Frame mapping and force feedback methods, devices and systems

• US 9,044,257 B2
• Filed: 10/08/2013
• Issued: 06/02/2015
• Est. Priority Date: 04/16/2007
• Status: Active Grant

• Current Assignee: Deerfield Imaging Incorporated
• Original Assignee: Deerfield Imaging SA RL
• Inventors: Fielding, Tim, Newhook, Perry
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): PECHE, JORGE O
• Application Number:
  US14/049,043
• Publication Number:
  US 20140142593A1
• Time in Patent Office: 602 Days
• Field of Search: 700/245, 700/247, 700/248, 700/251, 700/256, 700/259, 700/261, 700/264, 901/2, 901/5, 901/8, 901/9, 901/14, 901/16, 901/20, 901/46, 901/47
• US Class Current: 1/1
• CPC Class Codes: A61B 2034/305 Details of wrist mechanisms at distal ends of robotic arms
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/40137 Force sensation feedback from simulated tool
• CPC Class Codes: G05B 2219/40182 Master has different configuration than slave manipulator
• CPC Class Codes: G05B 2219/40401 Convert workspace of master to workspace of slave
• CPC Class Codes: G05B 2219/45117 Medical, radio surgery manipulator

Abstract

Methods, devices, and systems for controlling movement of a slave manipulator by an operator controlling a master manipulator in a way that the motion of that slave manipulator can be presented via a display to the operator such that the displayed position of the slave manipulator is intuitive to the operator, regardless of the actual position and orientation of the slave manipulator. Methods, devices, and systems relating to force feedback associated with medical robotic procedures.

Claims

1. A method for controlling movement of a robotic arm of a medical robot system, the robotic arm operatively associated with a tool, the medical robot system comprising:

a medical robot under operator control having at least one robotic arm,

a camera configured to provide an image of the tool, and

a reference structure having a fixed orientation relative to the camera, the method comprising;

determining a calculated motion of the tool wherein;

the determining of the calculated motion is based on an input value received from a controller device that provides input from a user; and

the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane defined by X and Y axis of an orthogonal coordinate system defined relative to the reference structure is parallel to and pointing in the same direction as the X axis; and

outputting an actuation signal to move the robotic arm in accordance with the calculated motion.

2. The method of claim 1, further comprising:

determining a calculated observer force value that is expressed in the observer coordinate system and based on a virtual force value; and

outputting a feedback signal to the controller device in accordance with the calculated observer force value.

3. The method of claim 1, further comprising:

determining a calculated observer force value that is expressed in the observer coordinate system and based on a measured force value from the robotic arm; and

outputting a feedback signal to the controller device in accordance with the calculated observer force value.

4. The method of claim 3, wherein the calculated observer force value is also based on a virtual force value.

5. The method of claim 4, wherein calculation of the virtual force value comprises using one or more calculated virtual collisions, one or more joint limits, motion divergence or a combination thereof.

6. The method of 1, further comprising:

determining a second calculated motion of a second tool of a second robotic arm, where;

the determining of the second calculated motion is based on a second input value received from a second controller device that provides input from the user; and

the second calculated motion is expressed in a second observer coordinate system that is based on the orientation of the second tool and the orientation of the observer reference structure; and

outputting a second actuation signal to move the second robotic arm in accordance with the second calculated motion.

7. The method of claim 6, further comprising:

determining a second calculated observer force value that is expressed in the second observer coordinate system and based on a second virtual force value; and

outputting a second feedback signal to the second controller device in accordance with the second calculated observer force value.

8. The method of claim 6, further comprising:

determining a second calculated observer force value that is expressed in the second observer coordinate system and based on a second measured force value from the second robotic arm; and

outputting a second feedback signal to the second controller device in accordance with the second calculated observer force value.

9. The method of claim 8, wherein the second calculated observer force value is also based on a second virtual force value.

10. The method of claim 9, wherein calculation of the second virtual force value comprises using one or more calculated virtual collisions, one or more joint limits, motion divergence, or a combination thereof.

11. A method for controlling movement of a robotic arm of a medical robot system, the robotic arm operatively associated with a tool, the medical robot system comprising:

a medical robot under operator control having at least one robotic arm, a camera configured to provide an image of the tool, and

a reference structure having a fixed orientation relative to the camera, the method comprising;

receiving an input value from a controller device representative of a desired motion of the tool, wherein the input value is expressed in a controller device coordinate system;

determining a calculated motion of the tool based on the input value, wherein the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane defined by X and Y axis of an orthogonal coordinate system defined relative to the reference structure is parallel to and pointing in the same direction as the X axis;

determining a commanded motion of the robotic arm based on the calculated motion of the tool and a position of the robotic arm;

outputting an actuation signal to move the robotic arm in accordance with the commanded motion;

determining a calculated observer force value that is expressed in the observer coordinate system;

determining a controller device feedback force value based on the calculated observer force value, wherein the controller device feedback force is expressed in the controller device coordinate system; and

outputting a feedback signal to the controller device in accordance with the controller device feedback force value.

12. The method of claim 11, where:

the input value can be expressed in the controller device coordinate system as an input vector having an input magnitude and an input direction;

the calculated motion can be expressed in the observer coordinate system as a calculated motion vector having a calculated motion magnitude and a calculated motion direction; and

the calculated motion direction equals the input direction.

13. The method of claim 12, wherein the determining of the calculated observer force value comprises using a virtual force value.

14. The method of claim 12, further comprising:

receiving a measured force value from the robotic arm, wherein the measured force value is representative of a force measured at a location on the robotic arm and expressed in a sensor coordinate system; and

where;

the determining of the calculated observer force value comprises using the measured force value and the position of the robotic arm.

15. The method of claim 14, wherein the determining of the calculated observer force value further comprises using a virtual force value.

16. The method of claim 15, wherein calculation of the virtual force value comprises using one or more calculated virtual collisions.

17. The method of claim 15, wherein calculation of the virtual force value comprises using one or more joint limits.

18. The method of claim 15, wherein calculation of the virtual force value comprises using motion divergence.

19. The method of claim 15, wherein calculation of the virtual force value comprises using one or more calculated virtual collisions, one or more joint limits, motion divergence, or a combination thereof.

20. A non-transitory computer readable storage medium comprising machine readable instructions for:

determining a calculated motion of a tool operatively associated with a robotic arm of a medical robot system, the medical robot system comprising a medical robot under operator control having at least one robotic arm, wherein;

the determining of the calculated motion is based on an input value received from a controller device that provides input from a user; and

the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane the plane being defined by X and Y axis of an orthogonal coordinate system defined relative to a reference structure that has a fixed orientation relative to a camera configured to provide an image of the tool, is parallel to and pointing in the same direction as the X axis; and

outputting an actuation signal to move the robotic arm in accordance with the calculated motion.

21. A non-transitory computer readable storage medium comprising machine readable instructions for:

receiving an input value from a controller device representative of a desired motion of a tool a tool operatively associated with a robotic arm of a medical robot system, the medical robot system comprising a medical robot under operator control having at least one robotic arm, wherein the input value is expressed in a controller device coordinate system;

determining a calculated motion of the tool based on the input value, wherein the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane, the plane being defined by X and Y axis of an orthogonal coordinate system defined relative to a reference structure having a fixed orientation relative to a camera configured to provide an image of the tool, is parallel to and pointing in the same direction as the X axis;

determining a commanded motion of the robotic arm based on the calculated motion of the tool and a position of the robotic arm;

outputting an actuation signal to move the robotic arm in accordance with the commanded motion;

determining a calculated observer force value that is expressed in the observer coordinate system;

determining a controller device feedback force value based on the calculated observer force value, wherein the controller device feedback force is expressed in the controller device coordinate system; and

outputting a feedback signal to the controller device in accordance with the controller device feedback force value.

22. A control system configured to control at least one robotic arm of a medical robot system, the robotic arm operatively associated with a tool, the medical robot system comprising:

a medical robot under operator control having at least one robotic arm,

a camera configured to provide an image of the tool, and

a reference structure having a fixed orientation relative to the camera,

the control system comprising;

a computing device operatively associated with the medical robot, wherein the computing device is configured to determine a calculated motion of the tool wherein the determination of the calculated motion is based on an input value received from a controller device that provides input from a user; and

the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane defined by X and Y axis of an orthogonal coordinate system defined relative to the reference structure is parallel to and pointing in the same direction as the X axis; and

output an actuation signal to move the robotic arm in accordance with the calculated motion, and

at least one controller device operatively associated with the computing device and configured to receive an input of a desired movement of the robotic arm.

23. A control system for use in configured to control of at least one medical robotic arm of a medical robot system, the robotic arm operatively associated with a tool, the medical robot system comprising:

a medical robot under operator control having at least one robotic arm,

a camera configured to provide an image of the tool, and

a reference structure having a fixed orientation relative to the camera, the control system comprising;

a computing device operatively associated with the medical robot, wherein the computing device is configured to receive an input value from a controller device representative of a desired motion of the tool, wherein the input value is expressed in a controller device coordinate system;

determine a calculated motion of the tool based on the input value, wherein the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane defined by X and Y axis of an orthogonal coordinate system defined relative to the reference structure is parallel to and pointing in the same direction as the X axis;

determine a commanded motion of the robotic arm based on the calculated motion of the tool and a position of the robotic arm;

output an actuation signal to move the robotic arm in accordance with the commanded motion;

determine a calculated observer force value that is expressed in the observer coordinate system;

determine a controller device feedback force value based on the calculated observer force value, wherein the controller device feedback force is expressed in the controller device coordinate system; and

output a feedback signal to the controller device in accordance with the controller device feedback force value, and

at least one controller device operatively associated with the computing device and configured to receive an input of a desired movement of the robotic arm.