SYSTEMS AND METHODS TO CORRECT FOR UNCOMMANDED INSTRUMENT ROLL

• US 20190167366A1
• Filed: 12/05/2018
• Published: 06/06/2019
• Est. Priority Date: 12/06/2017
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Ummalaneni, Ritwik, Jiang, Allen
• Granted Patent: US 10,987,179 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

Certain aspects relate to systems, methods, and techniques for correcting for uncommanded instrument roll. A method for adjusting a controller-feedback system in a medical instrument may comprise receiving data from a sensor at or near a distal end of the instrument, determining a tip frame of reference based on the data from the sensor, the tip frame of reference representing a current orientation of the distal end of the instrument, obtaining a desired frame of reference, determining an adjustment to a visual frame of reference or a control frame of reference based on the tip frame of reference and the desired frame of reference, and transforming the visual frame of reference or the control frame of reference.

Claims

1. A method for adjusting a controller-feedback system in a robotically controlled medical instrument inserted into a patient, the method comprising:

receiving data from at least one imaging device or location sensor at a distal end of the instrument;

determining a tip frame of reference based on the data from the at least one imaging device or location sensor, the tip frame of reference representing a current orientation of the distal end of the instrument;

obtaining a desired frame of reference representing a frame of reference relative to an anatomy frame of reference or a global frame of reference;

determining an adjustment to a visual frame of reference or a control frame of reference based on the tip frame of reference and the desired frame of reference; and

transforming the visual frame of reference or the control frame of reference based on the determined adjustment.

2. The method of claim 1, wherein obtaining the desired frame of reference is based on one or more pixel values of an image representing one or more anatomical features of the patient.

3. The method of claim 1, wherein obtaining the desired frame of reference is based on one or more anatomical features of a main carina of the patient.

4. The method of claim 1, wherein obtaining the desired frame of reference is based on data from one or more EM patches positioned on the patient.

5. The method of claim 1, wherein obtaining the desired frame of reference is based on the tip frame of reference.

6. The method of claim 1, wherein determining the adjustment is based on comparing between one or more features derived from at least one image of an anatomical feature and one or more features derived from at least one model of the anatomical feature.

7. The method of claim 6, wherein the at least one image of an anatomical feature is obtained from the imaging device at the distal end of the instrument.

8. The method of claim 1, wherein determining the adjustment is based on comparing between data from one or more EM sensors in the distal end of the instrument and data from one or more EM patches positioned on the patient.

9. The method of claim 1, wherein determining the adjustment is based on data from an accelerometer configured to measure force of gravity.

10. The method of claim 1, wherein transforming the visual frame of reference or the control frame of reference comprises rotating the visual frame of reference or the control frame of reference with respect to a longitudinal axis of the instrument.

11. The method of claim 1, wherein transforming the visual frame of reference or the control frame of reference comprises rotating the visual frame of reference or the control frame of reference to align with the tip frame of reference or the desired frame of reference.

12. The method of claim 1, wherein transforming the visual frame of reference or the control frame of reference is based on a user input.

13. The method of claim 1, further comprising verifying the transformed visual frame of reference or the transformed control frame of reference.

14. The method of claim 13, wherein verifying the transformed visual frame of reference or the transformed control frame of reference comprises:

moving the instrument in one direction;

calculating an expected change in the visual frame of reference or the control frame of reference in response to the movement of the instrument; and

comparing between (1) an actual change in the visual frame of reference or the control frame of reference and (2) the expected change.

15. A system configured to transform a control frame of a medical instrument configured to be inserted into a patient, the system comprising:

a control system configured to determine movement of the medical instrument;

at least one computer-readable memory having stored thereon executable instructions; and

one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the system to at least;

obtain a control frame of reference representing relationship between a motor control command and a motor output of the medical instrument;

determine a tip frame of reference based on data from at least one imaging device or location sensor at a distal end of the medical instrument, the tip frame of reference representing a current orientation of the distal end of the medical instrument;

obtain a desired frame of reference; and

transform the control frame of reference based on the tip frame of reference and the desired frame of reference.

16. The system of claim 15, wherein the at least one location sensor comprises an electromagnetic (EM) sensor.

17. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

receive the tip frame of reference from the medical instrument.

18. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

(1) receive data from a second sensor at the distal end of the medical instrument and (2) determine the tip frame of reference based on the data from the second sensor.

19. The system of claim 18, wherein the second sensor at the distal end of the instrument comprises at least one imaging device or location sensor.

20. The system of claim 19, wherein the second sensor comprises an electromagnetic (EM) sensor.

21. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

receive the control frame of reference from the control system.

22. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

(1) move the medical instrument in one direction and (2) determine the control frame of reference based on the movement of the medical instrument.

23. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

receive a visual frame of reference from the instrument and transform the visual frame of reference based on the tip frame of reference and the desired frame of reference.

24. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

(1) receive data from the at least one sensor at the distal end of the medical instrument, (2) determine a visual frame of reference based on the data from the at least one sensor, and (3) transform the visual frame of reference based on the tip frame of reference and the desired frame of reference.

25. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

determine the desired frame of reference based on one or more anatomical features of the patient.

26. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

determine the desired frame of reference based on one or more pixel values of an image representing the one or more anatomical features of the patient.

27. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

determine the desired frame of reference based on data from one or more EM patches positioned on the patient.

28. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

determine one or more differences between at least one image of an anatomical feature and at least one model of the anatomical feature; and

transform the control frame of reference based on the differences between the at least one image and the at least one model.

29. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

rotate the control frame of reference with respect to a longitudinal axis of the medical instrument to align with the desired frame of reference.

30. The system of claim 15, wherein the one or more processors in communication with the at least one computer-readable memory are configured to execute the instructions to cause the system to at least:

transform the control frame of reference based on a user input.