DETECTION OF MISALIGNMENT OF ROBOTIC ARMS

• US 20190107454A1
• Filed: 07/03/2018
• Published: 04/11/2019
• Est. Priority Date: 10/10/2017
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Lin, Jiayi, Chung, Shu-Yun
• Granted Patent: US 10,539,478 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 1/2736 Gastroscopes
• CPC Class Codes: A61B 1/307 for the urinary organs, e.g. urethroscopes, cystoscopes
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 2090/066 for measuring torque
• CPC Class Codes: A61B 2090/0811 for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument
• CPC Class Codes: A61B 2090/306 using optical fibres
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 46/10 specially adapted for instruments, e.g. microscopes
• CPC Class Codes: A61B 50/13 Trolleys, e.g. carts
• CPC Class Codes: A61B 6/032 : Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/0407 : Supports, e.g. tables or beds, for the body or parts of the body
• CPC Class Codes: A61B 90/06 : Measuring instruments not otherwise provided for
• CPC Class Codes: A61B 90/08 : Accessories or related features not otherwise provided for
• CPC Class Codes: B25J 9/163 : learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1676 : Avoiding collision or forbidden zones
• CPC Class Codes: G01L 5/226 : to manipulators, e.g. the force due to gripping
• CPC Class Codes: G05B 2219/39082 : Collision, real time collision avoidance
• CPC Class Codes: G05B 2219/40599 : Force, torque sensor integrated in joint

Abstract

Certain aspects relate to systems and techniques for detection of undesirable forces on one or more surgical robotic arms. In one aspect, there is provided a system including a robotic arm, including: two linkages, a joint, a torque sensor, and an instrument device manipulator (IDM). The system may further include a processor configured to measure a first torque value at the joint based on an output of the torque sensor and determine a second torque value at the joint based on a position of the robotic arm. The second torque value may be indicative of a gravitational component of the torque between the two linkages. The processor may be further configured to determine a force at the IDM based a difference between the first and second torque values and determine whether the robotic arm has collided with an object or misaligned based on the force at the IDM.

Claims

1. A system for detecting misalignment of robotic arms, comprising:

a first robotic arm comprising a first instrument device manipulator (IDM) connected to a distal end of the first robotic arm;

a second robotic arm comprising a second IDM connected to a distal end of the second robotic arm;

a first medical instrument comprising a working channel; and

a second medical instrument, the first robotic arm configured to drive the first medical instrument along a first axis, and the second robotic arm configured to drive the second medical instrument through the working channel of the first medical instrument;

a processor; and

a memory storing computer-executable instructions to cause the processor to;

determine a first force at the first IDM,

determine a second force at the second IDM,

determine that both of the first and second forces are greater than a threshold force, and

determine that the first and second robotic arms are misaligned in response to the processor determining that both of the first and second forces are greater than the threshold force.

2. The system of claim 1, wherein the memory further stores computer-executable instructions to cause the processor to:

determine based on insertion data that the second medical instrument was being driven through the first medical instrument at the time that the second force was detected,

wherein the determination that the first and second robotic arms are misaligned is further in response to the processor determining that the second medical instrument was being driven through the first medical instrument.

3. The system of claim 2, wherein the memory further stores computer-executable instructions to cause the processor to:

determine based on the insertion data that the first medical instrument was being driven at the time that the first and second IDMs experienced the first and second forces, respectively,

determine that the first force is greater than the threshold value, and

determine that the first robotic arm is misaligned with a patient introducer configured to guide the first medical instrument into a patient, in response to the processor determining that the first medical instrument was being driven at the time that the first and second IDMs experienced the first and second forces and determining that the first force is greater than the first threshold value.

4. The system of claim 2, wherein each of the first and second medical instruments comprises one of:

a sheath, a leader, a needle, forceps, and a brush.

5. The system of claim 1, wherein the memory further stores computer-executable instructions to cause the processor to:

determine that the first and second forces are in opposing directions, and

determine that a difference between the magnitudes of the first and second forces is less than a threshold difference,

wherein the determination that the first and second robotic arms are misaligned is further in response to the processor determining that the first and second forces are in opposing directions and that the difference between the magnitudes of the first and second forces is less than the threshold difference.

6. The system of claim 1, wherein each of the first and second robotic arms comprises:

at least two linkages,

at least one joint connecting the at least two linkages, and

at least one torque sensor configured to detect torque between the at least two linkages.

7. The system of claim 6, wherein the memory further stores computer-executable instructions to cause the processor to:

measure a first torque value at the at least one joint of the first robotic arm based on an output of the torque sensor,

determine a second torque value at the at least one joint of the first robotic arm based on a position of the first robotic arm, the second torque value indicative of a gravitational component of the torque between the at least two linkages,

determine a first force at the first IDM based on a difference between the first and second torque values, and

determine, based on the first force at the first IDM, whether a collision has occurred between the first robotic arm and an object.

8. The system of claim 7, wherein the memory further stores computer-executable instructions to cause the processor to:

encode an indication of the collision between the first robotic arm and the object, and

provide the encoded indication of the collision to a display configured to render encoded data.

9. The system of claim 7, wherein the memory further stores computer-executable instructions to cause the processor to:

determine a first component of the first force applied to the first IDM along a second axis perpendicular to the first axis, and

determine that the first component of the first force is greater than a first threshold value,

wherein the determining of the collision is further based on the processor determining that the first component of the first force is greater than the first threshold value.

10. The system of claim 9, wherein the memory further stores computer-executable instructions to cause the processor to:

determine a second component of the first force applied to the IDM along the first axis, and

determine that the second component of the first force is greater than a second threshold value,

wherein the determining of the collision is further based on the processor determining that the second component of the first force is greater than the second threshold value, the second threshold value greater than an expected force of insertion of the first medical instrument.

11. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:

determine a first force at a first IDM connected to a distal end a first robotic arm;

determine a second force at a second IDM connected to a distal end a second robotic arm, the first robotic arm configured to drive a first medical instrument comprising a working channel along a first axis, and the second robotic arm configured to drive a second medical instrument through the working channel of the first medical instrument;

determine that both of the first and second forces are greater than a threshold force; and

determine that the first and second robotic arms are misaligned in response to the determination that both of the first and second forces are greater than the threshold force.

12. The non-transitory computer readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine based on insertion data that the second medical instrument was being driven through the first medical instrument at the time that the second force was detected,

wherein the determination that the first and second robotic arms are misaligned is further in response to the determination that the second medical instrument was being driven through the first medical instrument.

13. The non-transitory computer readable storage medium of claim 12, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine based on the insertion data that the first medical instrument was being driven at the time that the first and second IDMs experienced the first and second forces, respectively;

determine that the first force is greater than the threshold value; and

determine that the first robotic arm is misaligned with a patient introducer configured to guide the first medical instrument into a patient in response to the determination that the first medical instrument was being driven at the time that the first and second IDMs experienced the first and second forces and determining that the first force is greater than the first threshold value.

14. The non-transitory computer readable storage medium of claim 12, wherein each of the first and second medical instruments comprise one of:

a sheath, a leader, a needle, forceps, and a brush.

15. The non-transitory computer readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine that the first and second forces are in opposing directions; and

determine that a difference between the magnitudes of the first and second forces is less than a threshold difference,

wherein the determination that the first and second robotic arms are misaligned is further in response to the determination that the first and second forces are in opposing directions and that the difference between the magnitudes of the first and second forces is less than the threshold difference.

16. The non-transitory computer readable storage medium of claim 11, wherein each of the first and second robotic arms comprises:

at least two linkages,

at least one joint connecting the at least two linkages, and

at least one torque sensor configured to detect torque between the at least two linkages.

17. The non-transitory computer readable storage medium of claim 16, further having stored thereon instructions that, when executed, cause at least one computing device to:

measure a first torque value at the at least one joint of the first robotic arm based on an output of the torque sensor;

determine a second torque value at the at least one joint of the first robotic arm based on a position of the first robotic arm, the second torque value indicative of a gravitational component of the torque between the at least two linkages;

determine a first force at the first IDM based on a difference between the first and second torque values; and

determine, based on the first force at the first IDM, whether a collision has occurred between the first robotic arm and an object.

18. The non-transitory computer readable storage medium of claim 17, further having stored thereon instructions that, when executed, cause at least one computing device to:

encode an indication of the collision between the first robotic arm and the object; and

provide the encoded indication of the collision to a display configured to render encoded data.

19. The non-transitory computer readable storage medium of claim 17, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine a first component of the first force applied to the first IDM along a second axis perpendicular to the first axis; and

determine that the first component of the first force is greater than a first threshold value,

wherein the determining of the collision is further based on determining that the first component of the first force is greater than the first threshold value.

20. The non-transitory computer readable storage medium of claim 19, further having stored thereon instructions that, when executed, cause at least one computing device to:

determine a second component of the first force applied to the IDM along the first axis; and

determine that the second component of the first force is greater than a second threshold value,

wherein the determining of the collision is further based on determining that the second component of the first force is greater than the second threshold value, the second threshold value greater than an expected force of insertion of the first medical instrument.

21. A method of positioning a first robotic arm, comprising:

determining a first force at a first IDM connected to a distal end of a first robotic arm;

determining a second force at a second IDM connected to a distal end of a second robotic arm, the first robotic arm configured to drive a first medical instrument comprising a working channel along a first axis, and the second robotic arm configured to drive a second medical instrument through the working channel of the first medical instrument;

determining that both of the first and second forces are greater than a threshold force; and

determining that the first and second robotic arms are misaligned in response to the determination that both of the first and second forces are greater than the threshold force.

22. The method of claim 21, further comprising:

determining, based on insertion data, that the second medical instrument was being driven through the first medical instrument at the time that the second force was detected,

wherein the determination that the first and second robotic arms are misaligned is further in response to the determination that the second medical instrument was being driven through the first medical instrument.

23. The method of claim 22, further comprising:

determining, based on the insertion data, that the first medical instrument was being driven at the time that the first and second IDMs experienced the first and second forces, respectively;

determining that the first force is greater than the threshold value; and

determining that the first robotic arm is misaligned with a patient introducer configured to guide the first medical instrument into a patient, in response to the determination that the first medical instrument was being driven at the time that the first and second IDMs experienced the first and second forces and that the first force is greater than the first threshold value.

24. The method of claim 22, wherein each of the first and second medical instruments comprise one of:

a sheath, a leader, a needle, forceps, and a brush.

25. The method of claim 21, further comprising:

determining that the first and second forces are in opposing directions; and

determining that a difference between the magnitudes of the first and second forces is less than a threshold difference,

wherein the determination that the first and second robotic arms are misaligned is further in response to the determination that the first and second forces are in opposing directions and that the difference between the magnitudes of the first and second forces is less than the threshold difference.

26. The method of claim 21, wherein each of the first and second robotic arms comprises:

at least two linkages,

at least one joint connecting the at least two linkages, and

at least one torque sensor configured to detect torque between the at least two linkages.

27. The method of claim 26, further comprising:

measuring a first torque value at the at least one joint of the first robotic arm based on an output of the torque sensor;

determining a second torque value at the at least one joint of the first robotic arm based on a position of the first robotic arm, the second torque value indicative of a gravitational component of the torque between the at least two linkages;

determining a first force at the first IDM based on a difference between the first and second torque values; and

determining, based on the first force at the first IDM, whether a collision has occurred between the first robotic arm and an object.

28. The method of claim 27, further comprising:

encoding an indication of the collision between the first robotic arm and the object; and

providing the encoded indication of the collision to a display configured to render encoded data.

29. The method of claim 26, further comprising:

determining a first component of the first force applied to the first IDM along a second axis perpendicular to the first axis; and

determining that the first component of the first force is greater than a first threshold value,

wherein the determining of the collision is further based on determining that the first component of the first force is greater than the first threshold value.

30. The method of claim 29, further comprising:

determining a second component of the first force applied to the IDM along the first axis; and

determining that the second component of the first force is greater than a second threshold value,

wherein the determining of the collision is further based on determining that the second component of the first force is greater than the second threshold value, the second threshold value greater than an expected force of insertion of the first medical instrument.