AUTOMATED ENDOSCOPE CALIBRATION

• US 20160374541A1
• Filed: 06/23/2016
• Published: 12/29/2016
• Est. Priority Date: 10/24/2014
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Surgical Robotics, Inc. (Johnson & Johnson)
• Inventors: Agrawal, Varun, Ghoreyshi, Atiyeh, Mintz, David S.
• Granted Patent: US 10,314,463 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/00057 provided with means for testing or calibration
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/0052 Constructional details of control elements, e.g. handles
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 34/30 Surgical robots

Abstract

A surgical robotic system automatically calibrates tubular and flexible surgical tools such as endoscopes. By accounting for nonlinear behavior of an endoscope, the surgical robotic system can accurately model motions of the endoscope and navigate the endoscope while performing a surgical procedure on a patient. The surgical robotic system models the nonlinearities using sets of calibration parameters determined based on images captured by an image sensor of the endoscope. Calibration parameters can describe translational or rotational movements of the endoscope in one or more axis, e.g., pitch and yaw, as well as a slope, hysteresis, or dead zone value corresponding to the endoscope'"'"'s motion. The endoscope can include tubular components referred to as a sheath and leader. An instrument device manipulator of the surgical robotic system actuates pull wires coupled to the sheath or the leader, which causes the endoscope to articulate.

Claims

1. A method for controlling an endoscope of a surgical robotic system, comprising:

receiving a command to move the endoscope using the surgical robotic system;

accessing a set of calibration parameters associated with the endoscope;

generating an adjusted command based on the command and the set of calibration parameters;

providing the adjusted command to the surgical robotic system to move the endoscope; and

receiving movement information describing motion of the endoscope in response at least to the adjusted command.

2. The method of claim 1, wherein the movement information is received from a strain gauge of an instrument device manipulator of the surgical robotic system.

3. The method of claim 1, wherein the set of calibration parameters describe nonlinear behavior of the endoscope and includes:

a first set of calibration parameters based on an image registration process using a plurality of images captured by an image sensor coupled to a tip of the endoscope; and

a second set of calibration parameters based on a calibration curve fitting process using previously recorded movement information of the endoscope.

4. The method of claim 3, wherein the nonlinear behavior is based at least in part on a stiffness and a length of a helix section of the endoscope.

5. The method of claim 3, wherein generating the adjusted command comprises:

modifying the command based on at least one calibration parameter from the first set; and

modifying the command based on at least one calibration parameter from the second set.

6. The method of claim 3, wherein the plurality of images includes information about a pattern on a surface of a dome-shaped calibration structure.

7. The method of claim 3, wherein image registration process using the plurality of images comprises:

generating, for each pair of images of the plurality of images, a difference array based on differences in pixel values between a first image of the pair and a second image of the pair;

generating, for each pair of images of the plurality of images, a gradient array based on rates of change in pixel values between a first image of the pair and a second image of the pair; and

wherein the first set of calibration parameters is based at least in part on the difference arrays and the gradient arrays.

8. The method of claim 7, wherein the image sensor includes a plurality of color channels and wherein the image registration process using the plurality of images further comprises applying a weighted average to the difference arrays and the gradient arrays based on a sensitivity of each of the color channels.

9. The method of claim 3, wherein the movement information includes at least:

a first segment describing motion of the endoscope in a forward direction along an axis; and

a second segment describing motion of the endoscope in a backward direction along the axis.

10. The method of claim 9, wherein the calibration curve fitting process using previously recorded movement information comprises:

generating a slope by averaging a first rate of change of position of the endoscope based on the first segment and a second rate of change of position of the endoscope based on the second segment;

generating a hysteresis by determining a difference between the first segment and the second segment; and

wherein the second set of calibration parameters includes the slope and the hysteresis.

11. The method of claim 1, wherein the endoscope in configured to move in one or more axes including at least a pitch axis and a yaw axis, and wherein the command includes at least a translational motion and a rotational motion.

12. A method for automated calibration of an endoscope of a surgical robotic system, comprising:

providing a command to move the endoscope using the surgical robotic system;

receiving movement information describing motion of the endoscope in response at least to the command;

receiving a plurality of images captured by an image sensor coupled to a tip of the endoscope during the motion of the endoscope in response at least to the command;

generating a first set of calibration parameters by performing image registration using the plurality of images;

generating a second set of calibration parameters by performing calibration curve fitting using the movement information; and

storing the first and the second sets of calibration parameters in association with the endoscope.

13. The method of claim 12, wherein performing the image registration using the plurality of images comprises:

generating, for each pair of images of the plurality of images, a difference array based on differences in pixel values between a first image of the pair and a second image of the pair;

generating, for each pair of images of the plurality of images, a gradient array based on rates of change in pixel values between a first image of the pair and a second image of the pair; and

wherein the first set of calibration parameters is based at least in part on the difference arrays and the gradient arrays.

14. The method of claim 13, wherein the image sensor includes a plurality of color channels and wherein performing the image registration using the plurality of images further comprises applying a weighted average to the difference arrays and the gradient arrays based on a sensitivity of each of the color channels.

15. The method of claim 12, wherein the movement information includes at least:

a first segment describing motion of the endoscope in a forward direction along an axis; and

a second segment describing motion of the endoscope in a backward direction along the axis.

16. The method of claim 15, wherein performing calibration curve fitting using the movement information comprises:

generating a slope by averaging a first rate of change of position of the endoscope based on the first segment and a second rate of change of position of the endoscope based on the second segment;

generating a hysteresis by determining a difference between the first segment and the second segment; and

wherein the second set of calibration parameters includes the slope and the hysteresis.

17. The method of claim 12, wherein storing the first and the second set of calibration parameters includes storing a unique identifier of the endoscope associated with the calibration parameters in a database storing information based on multiple endoscopes.

18. A surgical robotic system, comprising:

an endoscope;

a robotic arm including a drive mechanism, the drive mechanism coupled to the endoscope; and

a controller configured to;

receive a command to move the endoscope using the robotic arm;

access a set of calibration parameters associated with the endoscope;

generate an adjusted command based on the command and the set of calibration parameters; and

provide the adjusted command to the robotic arm to move the endoscope.

19. The system of claim 18, wherein the endoscope includes a sheath tubular component coupled to a leader tubular component, the leader tubular component including a helix section.

20. The system of claim 18, wherein the set of calibration parameters is based on an image registration process and calibration curve fitting process, and wherein the set of calibration parameters describe nonlinear behavior of the endoscope.