CONFIGURATION MARKER DESIGN AND DETECTION FOR INSTRUMENT TRACKING

• US 20100168763A1
• Filed: 04/23/2009
• Published: 07/01/2010
• Est. Priority Date: 12/31/2008
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: ZHAO, TAO, ZHAO, WENYI, NOWLIN, WILLIAM C.
• Granted Patent: US 9,867,669 B2
• Time in Patent Office: Days
• US Class Current: 606/130
• CPC Class Codes: A61B 2034/102 Modelling of surgical devices, implants or prosthesis
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 2090/3983 Reference marker arrangements for use with image guided surgery
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/94 coded with symbols, e.g. text

Abstract

The present disclosure relates to systems, methods, and tools for tool tracking using image-derived data from one or more tool located reference features. A method includes: directing illuminating light from a light source onto a robotic surgical tool within a patient body, wherein the tool includes a plurality of primitive features having known positions on the tool, and wherein each feature includes a spherical reflective surface; capturing stereo images of the plurality of primitive features when the tool is within the patient body, wherein the stereo images are captured by an image capture device adjacent the illumination source so that the illumination light reflected from the imaged primitive features toward the image capture device substantially aligns with spherical centers of the surfaces of the imaged primitive features, and determining a position for the tool by processing the stereo images so as to locate the spherical centers of the imaged primitive features by using the reflected light.

Claims

1. A surgical robotic tool tracking method comprising:

directing illuminating light from a light source onto a robotic surgical tool within a patient body, wherein the tool includes a plurality of primitive features having known positions on the tool, and wherein each feature includes a spherical reflective surface;

capturing stereo images of a plurality of the primitive features when the tool is within the patient body, wherein the stereo images are captured by a stereo image capture device adjacent the illumination source so that the illumination light reflected from the imaged primitive features toward the image capture device substantially aligns with spherical centers of the surfaces of the imaged primitive features; and

determining a position for the tool by processing the stereo images so as to locate the spherical centers of the imaged primitive features by using the reflected light.

2. A method as in claim 1, wherein the illuminating light includes a visible light spectrum.

3. A method as in claim 1, wherein determining a position for the tool by processing the image is accomplished so as to identify at least one of the primitive features by using specular reflected light.

4. A method as in claim 1, further comprising processing said stereo images so as to determine three-dimensional positional data for the spherical centers of the imaged primitive features.

5. A method as in claim 4, wherein a constellation algorithm is used to identify a pattern of the primitive features in the first image.

6. A method as in claim 5, further comprising:

generating an estimated tool state for the tool by using at least one prior tool state from a prior image of the tool or joint data from a robotic actuation system effecting movement of the tool; and

using the estimated tool state in the constellation algorithm.

7. A method as in claim 4, further comprising:

capturing stereo images for a plurality of time points;

generating an estimated tool state for the plurality of time points; and

rejecting any incompatible pattern detection by using a robust estimation technique.

8. A method as in claim 7, wherein the robust estimation technique used is a Random Sample Consensus (RANSAC) technique.

9. A method as in claim 4, wherein a model based image signature is used in the identification of a primitive feature in said images.

10. A method as in claim 4, further comprising:

processing the stereo images so as to identify a natural feature of the tool in both of said images;

determining a three-dimensional position for the identified natural feature; and

determining an image-derived tool state by using the three-dimensional position for the natural feature in combination with the three-dimensional positional data for the imaged primitive features.

11. A method as in claim 1, further comprising:

generating an estimated tool state for the tool by using at least one prior tool state from a prior image of the tool or joint data from a robotic actuation system effecting movement of the tool; and

using the estimated tool state to reject an incompatible pattern detection.

12. A method as in claim 1, wherein at least one of the primitive features comprises a convex or concave spherical reflective surface aligned with a joint axis of the tool, the surface being defined by a joint structure.

13. A robotic surgery system, comprising:

a robotic surgical tool having a plurality of primitive features having know positions on the tool, and wherein each feature comprises a spherical reflective surface;

a light source oriented to transmit illumination light within a patient body;

a stereo image capture device adjacent the illumination source so that the illumination light reflected from the primitive features toward the image capture device substantially aligns with a spherical centers of the spherical surfaces; and

a processor coupled with the image capture device and configured for determining a position for the tool by processing stereo images so as to locate the spherical centers of imaged primitive features by using the reflected light.

14. A system as in claim 13, further comprising a tangible medium comprising machine-readable instructions executable by said processor for processing said stereo images.

15. A system as in claim 13, wherein the processor is configured to determine a position for the tool by processing the stereo images so as to identify at least one of the primitive features by using specular reflected light.

16. A system as in claim 13, wherein one of the primitive features is aligned with a joint axis of the tool, the surface being defined by a joint structure.

17. A system as in claim 13, wherein the processor is configured so as to determine three-dimensional positional data for the spherical centers of the imaged primitive features by processing the stereo images.

18. A system as in claim 17, wherein the imaging device comprises a stereoscopic endoscope.

19. A system as in claim 13, wherein the spherical reflective surface comprises a convex or concave surface.

20. A system as in claim 13, wherein the plurality of features includes three or more features.

21. A system as in claim 20, wherein the plurality of features includes four or more features.

22. A surgical tool for use With a robotic surgery system, the surgery system including a stereo imaging device for capturing stereo images of the surgical tool during use and a processor coupled with the imaging device for processing the captured stereo images so as to determine image-derived positional information for the surgical tool, the surgical tool comprising a plurality of primitive features with each primitive feature comprising a spherical reflective surface.

23. An object tracking system, comprising:

an object having a plurality of primitive features with each primitive feature comprising a spherical reflective surface;

a light source oriented to transmit illumination light toward the object;

a stereo image capture device for capturing stereo images of said object, the image device being disposed adjacent the illumination source so that illumination light reflected from a plurality of primitive features toward the image capture device substantially aligns with spherical centers of the spherical surfaces, the image device outputting image data for said stereo images; and

a processor coupled with the image capture device and configured to process said image data so as to;

determine three-dimensional position data for three or more of the imaged primitive features; and

determine a position for the tool by processing the three-dimensional position data.

24. A method for estimating the pose of a surgical tool comprising three or more substantially corner-less primitive features having known positions on the tool, the method comprising:

using a stereoscopic endoscope to capture stereo images of three or more of the primitive features, the stereo images including a first image and a second image;

extracting at least three of the primitive features from the first image;

extracting at least three of the primitive features from the second image;

determining correspondences between extracted primitive features by using image signatures;

using the determined correspondences to determine three-dimension positions for at least three of the primitive features;

identifying a pattern in the extracted primitive features that corresponds to a pattern of the tool primitive features; and

estimating a pose for the surgical tool by using the identified pattern and the determined three-dimensional positions.