Detection of undesirable forces on a surgical robotic arm

• US 10,145,747 B1
• Filed: 10/10/2017
• Issued: 12/04/2018
• Est. Priority Date: 10/10/2017
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Lin, Jiayi, Chung, Shu-Yun
• Primary Examiner(s): Noori, Max
• Application Number:
  US15/729,569
• Time in Patent Office: 420 Days
• Field of Search: 73862041
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 1/2736 Gastroscopes
• CPC Class Codes: A61B 1/307 for the urinary organs, e.g. urethroscopes, cystoscopes
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 2090/066 for measuring torque
• CPC Class Codes: A61B 2090/0811 for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument
• CPC Class Codes: A61B 2090/306 using optical fibres
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 46/10 specially adapted for instruments, e.g. microscopes
• CPC Class Codes: A61B 50/13 Trolleys, e.g. carts
• CPC Class Codes: A61B 6/032 : Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/0407 : Supports, e.g. tables or beds, for the body or parts of the body
• CPC Class Codes: A61B 90/06 : Measuring instruments not otherwise provided for
• CPC Class Codes: A61B 90/08 : Accessories or related features not otherwise provided for
• CPC Class Codes: B25J 9/163 : learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1676 : Avoiding collision or forbidden zones
• CPC Class Codes: G01L 5/226 : to manipulators, e.g. the force due to gripping
• CPC Class Codes: G05B 2219/39082 : Collision, real time collision avoidance
• CPC Class Codes: G05B 2219/40599 : Force, torque sensor integrated in joint

Abstract

Certain aspects relate to systems and techniques for detection of undesirable forces on one or more surgical robotic arms. In one aspect, there is provided a system including a robotic arm, including: two linkages, a joint, a torque sensor, and an instrument device manipulator (IDM). The system may further include a processor configured to measure a first torque value at the joint based on an output of the torque sensor and determine a second torque value at the joint based on a position of the robotic arm. The second torque value may be indicative of a gravitational component of the torque between the two linkages. The processor may be further configured to determine a force at the IDM based a difference between the first and second torque values and determine whether the robotic arm has collided with an object or misaligned based on the force at the IDM.

Claims

1. A system, comprising:

a first robotic arm, comprising;

at least two linkages,

at least one joint connecting the at least two linkages,

at least one torque sensor configured to detect torque between the at least two linkages, and

an instrument device manipulator (IDM) connected to a distal end of the first robotic arm;

a processor; and

a memory storing computer-executable instructions to cause the processor to;

measure a first torque value at the at least one joint based on an output of the torque sensor,

determine a second torque value at the at least one joint based on a position of the first robotic arm, the second torque value indicative of a gravitational component of the torque between the at least two linkages,

determine a first force at the IDM based on a difference between the first and second torque values, and

determine whether the first robotic arm has collided with an object based on the first force at the IDM.

2. The system of claim 1, wherein the memory further comprises computer-executable instructions to cause the processor to:

encode an indication of the collision between the first robotic arm and the object in response to determining that the first robotic arm has collided with the object, and

provide the encoded indication of the collision to a display configured to render encoded data.

3. The system of claim 1, further comprising:

a steerable instrument configured to be driven within a patient by movement of the IDM along a first axis,

wherein the memory further comprises computer-executable instructions to cause the processor to;

determine a first component of the first force applied to the IDM along a second axis perpendicular to the first axis, and

determine that the first component of the first force is greater than a first threshold value,

wherein determining that the first robotic arm has collided with the object is further based on determining that the first component of the first force is greater than the first threshold value.

4. The system of claim 3, wherein the memory further comprises computer-executable instructions to cause the processor to:

determine a second component of the first force applied to the IDM along the first axis, and

determine that the second component of the first force is greater than a second threshold value,

wherein determining that the first robotic arm has collided with the object is further based on determining that the second component of the first force is greater than the second threshold value, the second threshold value greater than an expected force of insertion of the steerable instrument into the patient.

5. The system of claim 1, further comprising:

a steerable instrument configured to be driven within a patient, the steerable instrument comprising a first medical instrument and a second medical instrument configured to be driven through a working channel of the first medical instrument, the first robotic arm configured to drive the first medical instrument along a first axis; and

a second robotic arm configured to drive the second medical instrument through the first medical instrument,

wherein the memory further comprises computer-executable instructions to cause the processor to;

detect a second force at an IDM of the second robotic arm,

determine that both of the first and second forces is greater than a threshold force, and

determine that the first and second robotic arms are misaligned in response to determining that both of the first and second forces is greater than the threshold force.

6. The system of claim 5, wherein the memory further comprises computer-executable instructions to cause the processor to:

determine that insertion data indicates that the second medical instrument was being driven through the first medical instrument at the time that the second force was detected,

wherein determining that the first and second robotic arms are misaligned is further in response to determining that the insertion data indicates that the second medical instrument was being driven through the first medical instrument.

7. The system of claim 6, wherein the memory further comprises computer-executable instructions to cause the processor to:

determine that insertion data indicates that the first medical instrument was being driven into the patient at the time that the first and second torque values were measured,

determine that the first force is greater than the threshold value, and

determine that the first robotic arm is misaligned with a patient introducer configured to guide the first medical instrument into the patient in response to determining that the insertion data indicates that the first medical instrument was being driven and determining that the first force is greater than the first threshold value.

8. The system of claim 6, wherein each of the first and second medical instruments comprise one of:

a sheath, a leader, a needle, forceps, and a brush.

9. The system of claim 1, further comprising:

a steerable instrument configured to be driven within a patient, the steerable instrument comprising a first medical instrument and a second medical instrument configured to be driven through the first medical instrument, the first robotic arm configured to drive the first medical instrument along a first axis; and

a second robotic arm configured to drive the second medical instrument through the first medical instrument,

wherein the memory further comprises computer-executable instructions to cause the processor to;

detect a second force applied to an IDM of the second robotic arm,

determine that the first and second forces are in opposing directions,

determine that a difference between the magnitudes of the first and second forces is less than a threshold difference, and

determine that the first and second robotic arms are misaligned in response to determining that the first and second forces are in opposing directions and determining that the difference between the magnitudes of the first and second forces is less than the threshold difference.

10. The system of claim 1, wherein:

the robotic arm further comprises at least one position sensor configured to measure an angle between the at least two linkages,

the memory further comprises computer-executable instructions to cause the processor to determine the force based on an output of the position sensor,

the torque sensor, the motor, and the position sensor are located in the joint,

each of the torque sensor, the motor, and the position sensor is coupled to the two linkages connected to the joint.

11. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:

measure a first torque value at a joint of a first robotic arm based on an output of a torque sensor, the first robotic arm comprising;

two linkages connected by the joint, a torque sensor configured to detect torque between the two linkages, and an instrument device manipulator (IDM) connected to a distal end of the first robotic arm;

determine a second torque value at the joint based on a position of the first robotic arm, the second torque value indicative of a gravitational component of the torque between the two linkages;

determine a first force at the IDM based on a difference between the first and second torque values; and

determine whether the first robotic arm has collided with an object based on the first force at the IDM.

12. The non-transitory computer readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause at least one computing device to:

encode an indication of the collision between the first robotic arm and the object in response to determining that the first robotic arm has collided with the object, and

provide the encoded indication of the collision to a display configured to render encoded data.

13. The non-transitory computer readable storage medium of claim 11, wherein the first robotic arm is further configured to drive a steerable instrument within a patient by movement of the IDM along a first axis, the non-transitory computer readable storage medium further having stored thereon instructions that, when executed, cause at least one computing device to:

determine a first component of the first force applied to the IDM along a second axis perpendicular to the first axis;

determine that the first component of the first force is greater than a first threshold value; and

determine that the first robotic arm has collided with the object further based on determining that the first component of the first force is greater than the first threshold value.

14. The non-transitory computer readable storage medium of claim 13, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine a second component of the first force applied to the IDM along the first axis;

determine that the second component of the first force is greater than a second threshold value; and

determine that the first robotic arm has collided with the object further based on determining that the second component of the first force is greater than the second threshold value, the second threshold value greater than an expected force of insertion of the steerable instrument into the patient.

15. The non-transitory computer readable storage medium of claim 11, wherein the first robotic arm is further configured to drive a steerable instrument within a patient, the steerable instrument comprising a first medical instrument and a second medical instrument configured to be driven through a working channel of the first medical instrument by a second robotic arm, the first robotic arm configured to drive the first medical instrument along a first axis, the non-transitory computer readable storage medium further having stored thereon instructions that, when executed, cause at least one computing device to:

detect a second force at an IDM of the second robotic arm;

determine that both of the first and second forces is greater than a threshold force; and

determine that the first and second robotic arms are misaligned in response to determining that both of the first and second forces is greater than the threshold force.

16. The non-transitory computer readable storage medium of claim 15, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine that insertion data indicates that the second medical instrument was being driven through the first medical instrument at the time that the second force was detected; and

determine that the first and second robotic arms are misaligned further in response to determining that the insertion data indicates that the second medical instrument was being driven through the first medical instrument.

17. The non-transitory computer readable storage medium of claim 15, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine that insertion data indicates that the first medical instrument was being driven into the patient at the time that the first and second torque values were measured;

determine that the first force is greater than the threshold value; and

determine that the first robotic arm is misaligned with a patient introducer configured to guide the first medical instrument into the patient in response to determining that the insertion data indicates that the first medical instrument was being driven and determining that the first force is greater than the first threshold value.

18. The non-transitory computer readable storage medium of claim 15, wherein each of the first and second medical instruments comprise one of:

a sheath, a leader, a needle, forceps, and a brush.

19. The non-transitory computer readable storage medium of claim 11, wherein the first robotic arm is further configured to drive a steerable instrument within a patient, the steerable instrument comprising a first medical instrument and a second medical instrument configured to be driven through a working channel of the first medical instrument by a second robotic arm, the first robotic arm configured to drive the first medical instrument along a first axis, the non-transitory computer readable storage medium further having stored thereon instructions that, when executed, cause at least one computing device to:

detect a second force applied to an IDM of the second robotic arm;

determine that the first and second forces are in opposing directions;

determine that a difference between the magnitudes of the first and second forces is less than a threshold difference; and

determine that the first and second robotic arms are misaligned in response to determining that the first and second forces are in opposing directions and determining that the difference between the magnitudes of the first and second forces is less than the threshold difference.

20. The non-transitory computer readable storage medium of claim 11, wherein:

the robotic arm further comprises at least one position sensor configured to measure an angle between the at least two linkages,

the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause at least one computing device to determine the force based on an output of the position sensor,

the torque sensor, the motor, and the position sensor are located in the joint, and

each of the torque sensor, the motor, and the position sensor is coupled to the two linkages connected to the joint.

21. A method of positioning a first robotic arm, comprising:

measuring a first torque value at a joint of a first robotic arm based on an output of a torque sensor, the first robotic arm comprising;

two linkages connected by the joint, a torque sensor configured to detect torque between the two linkages, and an instrument device manipulator (IDM) connected to a distal end of the first robotic arm;

determining a second torque value at the joint based on a position of the first robotic arm, the second torque value indicative of a gravitational component of the torque between the two linkages;

determining a first force at the IDM based on a difference between the first and second torque values; and

determining whether the first robotic arm has collided with an object based on the first force at the IDM.

22. The method of claim 21, further comprising:

encoding an indication of the collision between the first robotic arm and the object in response to determining that the first robotic arm has collided with the object, and

providing the encoded indication of the collision to a display configured to render encoded data.

23. The method of claim 21, wherein the first robotic arm is further configured to drive a steerable instrument within a patient by movement of the IDM along a first axis, the method further comprising:

determining a first component of the first force applied to the IDM along a second axis perpendicular to the first axis;

determining that the first component of the first force is greater than a first threshold value; and

determining that the first robotic arm has collided with the object further based on determining that the first component of the first force is greater than the first threshold value.

24. The method of claim 23, further comprising:

determining a second component of the first force applied to the IDM along the first axis;

determining that the second component of the first force is greater than a second threshold value; and

determining that the first robotic arm has collided with the object further based on determining that the second component of the first force is greater than the second threshold value, the second threshold value greater than an expected force of insertion of the steerable instrument into the patient.

25. The method of claim 21, wherein the first robotic arm is further configured to drive a steerable instrument within a patient, the steerable instrument comprising a first medical instrument and a second medical instrument configured to be driven through a working channel of the first medical instrument by a second robotic arm, the first robotic arm configured to drive the first medical instrument along a first axis, the method further comprising:

detecting a second force at an IDM of the second robotic arm;

determining that both of the first and second forces is greater than a threshold force; and

determining that the first and second robotic arms are misaligned in response to determining that both of the first and second forces is greater than the threshold force.

26. The method of claim 25, further comprising:

determining that insertion data indicates that the second medical instrument was being driven through the first medical instrument at the time that the second force was detected; and

determining that the first and second robotic arms are misaligned further in response to determining that the insertion data indicates that the second medical instrument was being driven through the first medical instrument.

27. The method of claim 25, further comprising:

determining that insertion data indicates that the first medical instrument was being driven into the patient at the time that the first and second torque values were measured;

determining that the first force is greater than the threshold value; and

determining that the first robotic arm is misaligned with a patient introducer configured to guide the first medical instrument into the patient in response to determining that the insertion data indicates that the first medical instrument was being driven and determining that the first force is greater than the first threshold value.

28. The method of claim 25, wherein each of the first and second medical instruments comprise one of:

a sheath, a leader, a needle, forceps, and a brush.

29. The method of claim 21, wherein the first robotic arm is further configured to drive a steerable instrument within a patient, the steerable instrument comprising a first medical instrument and a second medical instrument configured to be driven through a working channel of the first medical instrument by a second robotic arm, the first robotic arm configured to drive the first medical instrument along a first axis, the method further comprising:

detecting a second force applied to an IDM of the second robotic arm;

determining that the first and second forces are in opposing directions;

determining that a difference between the magnitudes of the first and second forces is less than a threshold difference; and

determining that the first and second robotic arms are misaligned in response to determining that the first and second forces are in opposing directions and determining that the difference between the magnitudes of the first and second forces is less than the threshold difference.

30. The method of claim 21, wherein:

the robotic arm further comprises at least one position sensor configured to measure an angle between the at least two linkages,

the method further comprises determining the force based on an output of the position sensor,

the torque sensor, the motor, and the position sensor are located in the joint, and

each of the torque sensor, the motor, and the position sensor is coupled to the two linkages connected to the joint.