A Supervised Autonomous Robotic System for Complex Surface Inspection and Processing

• US 20160016312A1
• Filed: 03/17/2014
• Published: 01/21/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: Herman, Herman, Stentz, Anthony, Lawrence, Stuart Edwin III, Baker, Christopher L, Baker, Christopher Randolph, Galati, David G, Haines, Justin C, Kelley, Alonzo J, Meyhofer, Eric, Valois, Jean-Sebastien
• Granted Patent: US 9,796,089 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B23Q 17/2233 for adjusting the tool relative to the workpiece
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01N 21/9515 Objects of complex shape, e.g. examined with use of a surface follower device measuring contours and curvatures G01B11/24
• CPC Class Codes: G05B 19/4099 Surface or curve machining, making 3D objects, e.g. desktop manufacturing
• CPC Class Codes: G05B 2219/37449 Inspection path planner
• CPC Class Codes: G05B 2219/45071 Aircraft, airplane, ship cleaning manipulator, paint stripping
• CPC Class Codes: Y10S 901/06 Communication with another machine
• CPC Class Codes: Y10S 901/28 Joint
• CPC Class Codes: Y10S 901/41 Tool
• CPC Class Codes: Y10S 901/44 inspection
• CPC Class Codes: Y10S 901/47 Optical

Abstract

The invention disclosed herein describes a supervised autonomy system designed to precisely model, inspect and process the surfaces of complex three-dimensional objects. The current application context for this system is laser coating removal of aircraft, but this invention is suitable for use in a wide variety of applications that require close, precise positioning and maneuvering of an inspection or processing tool over the entire surface of a physical object. For example, this system, in addition to laser coating removal, could also apply new coatings, perform fine-grained or gross inspection tasks, deliver and/or use manufacturing process tools or instruments, and/or verify the results of other manufacturing processes such as but not limited to welding, riveting, or the placement of various surface markings or fixtures.

Claims

1. A method of performing surface processing on a three dimensional object using a robotic manipulator, with a stationary or mobile base, in a work environment comprising the steps of:

obtaining a base model of said three dimensional object;

generating a surface model from said base model by performing a surface scan of said three dimensional object and augmenting said base model with range data and surface property data obtained from said scan;

planning a temporal sequence of states of said robotic manipulator to maximize processing of said surface of said three dimensional object;

scanning said surface of said three dimensional model before said surface is processed to control or modulate a processing tool connected to said robotic manipulator;

processing said surface with said processing tool.

scanning said surface of said three dimensional model as said surface is being processed and updating said surface model with new surface property data;

adjusting said set of positions and poses for said robotic manipulator based on said updated surface model; and

contintinuing processing of said surface and said updating of said surface model until said surface reaches a desired state.

2. The method of claim 1 further comprising the steps of:

refining the position of said surface model within said work environment; and

refining the position of said robotic manipulator within said work environment.

3. The method of claim 1 wherein said robotics manipulator is mounted on a mobile base which is capable of moving on a surface.

4. The method of claim 1 wherein said robotics manipulator is mounted on a rail base capable of moving said robotics manipulator along said rail.

5. The method of claim 1, wherein said base model is obtained from a pre-existing 3-D model with subsurface properties.

6. The method of claim 1, wherein said base model is generated by scanning said three dimensional object using an optical sensor capable of capturing the shape of said three dimensional object.

7. The method of claim 1 further comprising the step of augmenting said base model with processing policies regarding the processing of said surface.

8. The method of claim 7 further comprising the step of allowing user interaction with said surface model.

9. The method of claim 8 wherein said user interaction includes allowing the specification of virtual masks which preclude portions of said surface from being processed.

10. The method of claim 8 wherein said user interaction includes allowing the specification of virtual keep out areas or volumes which prevents the positioning of said robotic manipulator over said specified area or inside said specified volume.

11. The method of claim 8 wherein said user interaction includes allowing the marking of an area that needs further processing

12. The method of claim 8 wherein said user interaction allows the overriding of said processing policies.

13. The method of claim 1 wherein said robotic manipulator includes a LIDAR device and further wherein said range data is generated by said LIDAR device.

14. The method of claim 1 wherein said robotic manipulator includes one or more sensors, and further wherein said surface property data is generated by said one or more sensors.

15. The method of claim 14 wherein said one or more sensors is selected from a group comprising multi-spectral cameras, color cameras, monochrome cameras and cameras with specific wavelength sensitivities.

16. The method of claim 14 wherein said one or more sensors is selected from a group comprising inspection sensors, including non-destructive inspection sensors, ultrasonic sensors, and eddy current based crack detection sensors.

17. The method of claim 1 wherein said temporal sequence of states of said robotic manipulator comprises a set of positions and poses.

18. The method of claim 1 wherein said step of planning a temporal sequence of states of said robotic manipulator includes maximizing the processing of said surface while avoiding a collision between said three dimensional object and other objects in said work environment.

19. The method of claim 1 wherein said step of planning a temporal sequence of states of said robotic manipulator precludes processing the same area of said surface more than once within a predetermined minimum period of time.

20. The method of claim 1 wherein said surface model consists of a set of two dimensional manifolds, each of said manifolds composed of a plurality of cells, each of said cells representing a physical unit of surface area and each of said cells having a set of properties associated therewith.

21. The method of claim 20 where said properties associated with each of said cells includes the type of material and coating properties.

22. The method of claim 1 wherein said processing tool is a laser capable of ablating paint from a surface.

23. The method of claim 1 further comprising the steps of:

querying the most recently observed state of said surface along a trajectory of said processing tool; and

planning a sequence of processing actions and parameters to be carried out by said processing tool based on said observed state.

24. The method of claim 1 further comprising the step of analyzing a processed surface to determine if further processing is necessary for any portion of said surface.

25. The method of claim 1 wherein said robotic manipulator includes an end effector, further comprising the step of measuring the 3-D position and orientation of said end effector relative to a work area using an optical positioning camera viewing fiducials in said work area, said optical positioning camera being mounted close to said end effector to minimize pose error due to relative motion between optical based positioning and said robotic manipulator.

26. The method of claim 1 wherein said robotic manipulator includes an end effector, further comprising the step of measuring the 3-D position and orientation of said end effector using an optical positioning camera sighting said surface to measure relative motion between said end effector and said surface.

27. A system for performing surface processing on a three dimensional object in a work environment comprising:

one or more robotic manipulators, each of said robotic manipulators including an end effector having a surface processing tool, a ranging sensor and one or more surface property sensors attached thereto;

a computer running software for controlling said one or more robotic manipulators, said software comprising;

a surface property analyzer, for processing surface properties sensor data before and after the processing point of said processing tool to classify the current state of said surface;

a surface model for storage of the geometry of said surface and said current state of said surface;

a surface process planner for planning a sequence of processing actions to be carried out by said processing tool based on an observed state of said surface stored in state surface model;

a surface process modulator for modifying said planned sequence of processing actions based on real-time feedback from said ranging sensor and said one or more surface property sensors; and

a surface coverage planner, for planning movements and poses of said one or more robotic manipulators to maximize coverage of said surface.

28. The system of claim 27 wherein said processing tool is a laser capable of ablating paint from said surface.

29. The system of claim 27 further comprising one or more sensors for detecting and locating objects surrounding said system to prevent collision with said objects.

30. The system of claim 27 wherein said one or more robotic manipulators are mounted on a mobile base capable of moving on a surface.

31. The method of claim 27 wherein said one or more robotic manipulators are mounted on a rail base capable of moving said one or more robotic manipulators along the rail.

32. The system of claim 27 wherein said ranging sensor selected from a group comprising a scanning laser rangefinder, a stereo camera, a range camera and a range capture sensor.

33. The system of claim 27 wherein said one or more surface property sensors are selected from a group comprising video cameras and inspection sensors.

34. The system of claim 27 further comprising active illumination to illuminate the area viewed by said one or more surface property sensors.

35. The system of claim 27 wherein said software allows manual editing of said surface model by a user to effect changes in said sequence of processing actions.

36. The system of claim 27 wherein said surface is the fuselage, wings and control surfaces of an aircraft and further wherein said surface processing is the removal of coating layers from said surface.