Frame mapping and force feedback methods, devices and systems

• US 8,554,368 B2
• Filed: 04/16/2008
• Issued: 10/08/2013
• Est. Priority Date: 04/16/2007
• Status: Active Grant

• Current Assignee: Deerfield Imaging Incorporated
• Original Assignee: Governors of the University of Calgary
• Inventors: Fielding, Tim, Newhook, Perry
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): PECHE, JORGE O
• Application Number:
  US12/596,417
• Publication Number:
  US 20110160745A1
• Time in Patent Office: 2,001 Days
• Field of Search: 700/245, 700/247, 700/248, 700/251, 700/256, 700/259, 700/261, 700/264, 901/2, 901/5, 901/8, 901/9, 901/14, 901/16, 901/20, 901/46, 901/47
• US Class Current: 700/247
• CPC Class Codes: A61B 2034/305 Details of wrist mechanisms at distal ends of robotic arms
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/40137 Force sensation feedback from simulated tool
• CPC Class Codes: G05B 2219/40182 Master has different configuration than slave manipulator
• CPC Class Codes: G05B 2219/40401 Convert workspace of master to workspace of slave
• CPC Class Codes: G05B 2219/45117 Medical, radio surgery manipulator

Abstract

Methods, devices, and systems for controlling movement of a slave manipulator by an operator controlling a master manipulator in a way that the motion of that slave manipulator can be presented via a display to the operator such that the displayed position of the slave manipulator is intuitive to the operator, regardless of the actual position and orientation of the slave manipulator. Methods, devices, and systems relating to force feedback associated with medical robotic procedures.

Claims

1. A method for controlling movement of a robotic arm of a medical robot system, the robotic arm operatively associated with a tool, the medical robot system comprising:

a medical robot under operator control having at least one robotic arm,

a camera configured to provide an image of the tool, and

a reference structure having a fixed orientation relative to the camera, the method comprising;

a. receiving an input value from a controller device representative of a desired motion of the tool, wherein the input value is expressed in a controller device coordinate system;

b. determining a calculated motion of the tool based on the input value,

wherein the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane defined by X and Y axes of an orthogonal coordinate system defined relative to the reference structure is parallel to and pointing in the same direction as the X axis;

c. determining a commanded motion of the robotic arm based on the calculated motion and the position of the robotic arm; and

d. outputting an actuation signal to move the robotic arm in accordance with the commanded motion.

2. The method of claim 1, wherein:

the input value can be expressed in the controller device coordinate system as an input vector having an input magnitude and an input direction;

the calculated motion can be expressed in the observer coordinate system as a calculated motion vector having a calculated motion magnitude and a calculated motion direction; and

the calculated motion direction equals the input direction.

3. The method of claim 1, further comprising:

determining a calculated observer force value that is expressed in the observer coordinate system and based on a virtual force value; and

outputting a feedback signal to the controller device in accordance with the calculated observer force value.

4. The method of claim 3, wherein calculation of the virtual force value comprises using one or more calculated virtual collisions, one or more joint limits, motion divergence, or a combination thereof.

5. The method of claim 1, further comprising:

determining a calculated observer force value that is expressed in the observer coordinate system and based on a measured force value from the robotic arm; and

outputting a feedback signal to the controller device in accordance with the calculated observer force value.

6. The method of claim 5, wherein the measured force value is representative of a force measured at a location on the robotic arm and expressed in a sensor coordinate system and wherein the determining of the calculated observer force value is based on the measured force value and the position of the robotic arm.

7. The method of claim 5, wherein the calculated observer force value is also based on a virtual force value.

8. The method of claim 1, wherein the medical robot comprises two robotic arms, each operatively associated with a tool, and the method further comprises:

receiving a second input value from a second controller device representative of a second desired motion of a second tool, wherein the second input value is expressed in a second controller device coordinate system;

determining a second calculated motion of a second tool based on the second input value, wherein the second calculated motion is expressed in a second observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the second tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane defined by X and Y axes of the orthogonal coordinate system defined relative to the reference structure is parallel to and pointing in the same direction as the X axis;

determining a second commanded motion of the second robotic arm based on the second calculated motion and the position of the second robotic arm; and

outputting a second actuation signal to move the second robotic arm in accordance with the second commanded motion.

9. The method of claim 8, further comprising:

determining a second calculated observer force value that is expressed in the second observer coordinate system and based on a second virtual force value, and

outputting a second feedback signal to the controller device in accordance with the second calculated observer force value.

10. The method of claim 9, wherein calculation of the second virtual force value comprises using one or more calculated virtual collisions, one or more joint limits, motion divergence, or a combination thereof.

11. The method of claim 8, further comprising:

determining a second calculated observer force value that is expressed in the second observer coordinate system and based on a second measured force value from the second robotic arm, and

outputting a second feedback signal to the controller device in accordance with the second calculated observer force value.

12. The method of claim 11, wherein the second measured force value is representative of a force measured at a location on the second robotic arm and expressed in a sensor coordinate system and wherein the determining of the second calculated observer force value is based on the second measured force value and the position of the second robotic arm.

13. The method of claim 11, wherein the second calculated observer force value is also based on a second virtual force value.

14. The method of claim 1, wherein the observer coordinate system further comprises a third axis orthogonal to both the first axis and the second axis.

15. The method of claim 14, wherein the controller device coordinate system comprises three orthogonal axes, and wherein the input value received from the controller device is directly mapped to the calculated motion of the tool.

16. The method of claim 14, wherein the controller device coordinate system comprises three orthogonal axes, and wherein each axis of the controller device coordinate system, viewed from an operator perspective, is identically oriented to a corresponding axis of the observer coordinate system, viewed from a perspective of the reference structure.

17. A computer readable medium comprising machine readable instructions for performing the method according to claim 1.

18. A control system configured to control movement of at least one robotic arm of a medical robot system, the robotic arm operatively associated with a tool, the medical robot system comprising:

a medical robot under operator control having at least one robotic arm,

a camera configured to provide an image of the tool, and

a reference structure having a fixed orientation relative to the camera,

the control system comprising;

a computing device operatively associated with the medical robot, wherein the computing device is configured to receive an input value from a controller device representative of a desired motion of the tool, wherein the input value is expressed in a controller device coordinate system;

determine a calculated motion of the tool based on the input value, wherein the calculated motion is expressed in an observer coordinate system that comprises a first axis oriented in fixed relation with an axis of the tool and a second axis oriented orthogonal to the first axis and such that its projection into a plane defined by X and Y axes of an orthogonal coordinate system defined relative to the reference structure is parallel to and pointing in the same direction as the X axis;

determine a commanded motion of the robotic arm based on the calculated motion and the position of the robotic arm; and

output an actuation signal to move the robotic arm in accordance with the commanded motion, and

at least one controller device operatively associated with the computing device and configured to receive an input of a desired movement of the robotic arm.

19. The control system according to claim 18, further comprising a viewing instrument configured to display the image of the tool.