DETERMINING POSITION OF MEDICAL DEVICE IN BRANCHED ANATOMICAL STRUCTURE

• US 20140180063A1
• Filed: 10/08/2013
• Published: 06/26/2014
• Est. Priority Date: 10/12/2012
• Status: Abandoned Application

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Zhao, Tao, Prisco, Giuseppe Maria
• Application Number:
  US14/048,331
• Publication Number:
  US 20140180063A1
• Time in Patent Office: Days
• US Class Current: 600/424
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/009 with bending or curvature detection of the insertion part
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/0059 using light, e.g. diagnosis by transillumination, diascopy, fluorescence A61B5/0093, A61B5/1455, A61B5/02416 and A61B5/0261 take precedence
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 6/5247 combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 8/0841 for locating instruments
• CPC Class Codes: A61B 8/5261 combining images from different diagnostic modalities, e.g. ultrasound and X-ray
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/75 involving models

Abstract

Information extracted from sequential images captured from the perspective of a distal end of a medical device moving through an anatomical structure are compared with corresponding information extracted from a computer model of the anatomical structure. A most likely match between the information extracted from the sequential images and the corresponding information extracted from the computer model is then determined using probabilities associated with a set of potential matches so as to register the computer model of the anatomical structure to the medical device and thereby determine the lumen of the anatomical structure which the medical device is currently in. Sensor information may be used to limit the set of potential matches. Feature attributes associated with the sequence of images and the set of potential matches may be quantitatively compared as part of the determination of the most likely match.

Claims

1. A medical system comprising:

a memory storing information of a computer model of a branched anatomical structure; and

a processor programmed to register the computer model to a medical device for determining a position of the medical device in the branched anatomical structure by determining a most likely match between information which has been extracted from a sequence of images that has been captured by an image capturing device from a perspective of a distal end of the medical device while moving through a path including a plurality of lumens in the branched anatomical structure and corresponding information that has been extracted from the computer model of the branched anatomical structure.

2. The medical system of claim 1, wherein the information extracted from the sequence of images comprises information of feature points appearing in the sequence of images.

3. The medical system of claim 1, wherein the information extracted from the sequence of images comprises information of lumens appearing in the sequence of images.

4. The medical system of claim 3, further comprising:

extracting blobs from the sequence of captured images so that each lumen appearing in the sequence of images is indicated by one of the extracted blobs and the information of lumens appearing in the sequence of images comprises information of the extracted blobs.

5. The medical system of claim 4, wherein the information of the extracted blobs comprises information of feature attributes of the extracted blobs.

6. The medical system of claim 4, wherein the processor is programmed to identify the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets by:

determining information of feature attributes of the extracted blobs; and

comparing the feature attributes with corresponding feature attributes extracted from the computer model of the branched anatomical structure.

7. The medical system of claim 6, wherein the determining information of feature attributes of the extracted blobs comprises determining information of one or more of:

the number, shapes, sizes, locations, and orientations of the extracted blobs.

8. The medical system of claim 7 wherein the determining information of the feature attributes of the extracted blobs comprises:

determining at least one of one or more ratios of dimensions related to the extracted blobs and one or more angles between lines extending through centroids of pairs of extracted blobs, wherein each pair of extracted blobs indicate lumens that branch out of a same node in the branched anatomical structure.

9. The medical system of claim 8, wherein the identifying of the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets comprises restricting the set of potential matches to nodes corresponding to synthetic images that are extractable from the computer model of the branched anatomical structure and have the same number and relative positions of lumens appearing in the synthetic images as in the tracklet, wherein the relative positions indicate which lumens appear within other lumens.

10. The medical system of claim 9, wherein the identifying of the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets comprises restricting the set of potential matches to nodes corresponding to synthetic images that are extractable from the computer model of the branched anatomical structure and have feature attributes that are within a threshold difference to feature attributes of the extracted blobs of the tracklet.

11. The medical system of claim 10, wherein the identifying of the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets comprises generating quantitative comparisons of the feature attributes of the extracted blobs of the tracklet to corresponding feature attributes of the synthetic images of members of the set of potential matches by determining a value indicating a difference between one of the feature attributes in the extracted blobs and a corresponding one of the feature attributes in the synthetic images.

12. The medical system of claim 1, wherein the sequence of images comprises a plurality of tracklets, wherein each of the plurality of tracklets comprises a subset of the sequence of images which is topologically equivalent, but topologically different than adjacent ones of the plurality of tracklets, and wherein the processor is programmed to determine the most likely match by:

identifying a set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets;

determining a probability indicating a quality of match for each member of the set of potential matches for each of the plurality of tracklets; and

determining a maximal combined probability using information of the probabilities for the set of potential matches for each of the plurality of tracklets.

13. The medical system of claim 12, wherein the processor is further programmed to determine the most likely match by:

determining a set of transitional probabilities for each member of the set of potential matches for each of the plurality of tracklets, wherein each member of the set of transitional probabilities indicates a quality of match for a transition from the tracklet to a next in sequence tracklet of the plurality of tracklets; and

determining the maximal combined probability using information of the probabilities and transitional probabilities for the set of potential matches for each of the plurality of tracklets.

14. The medical system of claim 13, wherein the processor is programmed to determine the set of transitional probabilities for each member of the set of potential matches for each of the plurality of tracklets using static knowledge of connectivity in the computer model of the branched anatomical structure.

15. The medical system of claim 13, wherein the processor is programmed to determine the set of transitional probabilities for each member of the set of potential matches for each of the plurality of tracklets using dynamic knowledge of the movement of the medical device through the path in the branched anatomical structure.

16. The medical system of claim 13, wherein the processor is programmed to identify the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets by only including branches of the computer model of the branched anatomical structure which are topologically equivalent to the tracklet.

17. The medical system of claim 13, wherein a same set of lumens appears in each tracklet and wherein adjacent tracklets have at least one lumen in common.

18. The medical system of claim 17, wherein the processor is programmed to identify each tracklet by extracting and tracking blobs in the sequence of images until a set of the blobs changes, wherein each of the blobs corresponds to a lumen within the branched anatomical structure.

19. The medical system of claim 17, wherein the processor is programmed to determine membership of extracted blobs for each tracklet by ignoring false additions and subtractions from the set of extracted blobs in the sequence of images.

20. The medical system of claim 19, wherein the processor is programmed to analyze the set of sequential images to detect falsely identified lumens in the extracted blobs which would result in false additions to the set of extracted blobs and to detect a false determination of the medical device having entered a lumen which would result in a false subtraction from the set of extracted blobs.

21. The medical system of claim 13, wherein the processor is programmed to determine information of feature attributes of the extracted blobs by filtering the determined information by using at least one of:

averaging the determined information and eliminating outliers in the determined information.

22. The medical system of claim 13, wherein the computer model of the branched anatomical structure is spatially registered to the branched anatomical structure which the medical device is moving through, the medical system further comprising:

a position sensor for determining a position of the distal end of the medical device relative to the branched anatomical structure;

wherein the processor is programmed to identify the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets by using position information from the position sensor.

23. The medical system of claim 13, wherein the computer model of the branched anatomical structure is spatially registered to the branched anatomical structure which the medical device is moving through, the medical system further comprising:

an orientation sensor for determining an orientation of the distal end of the medical device relative to a fixed horizontal plane;

wherein the processor is programmed to identify the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets by using orientation information from the orientation sensor.

24. The medical system of claim 13, wherein the computer model of the branched anatomical structure is spatially registered to the branched anatomical structure which the medical device is moving through, the medical system further comprising:

a roll sensor for determining a roll angle of the medical device about a longitudinal axis of the medical device;

wherein the processor is programmed to identify the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets by using roll information from the roll sensor.

25. The medical system of claim 13, wherein the computer model of the branched anatomical structure is spatially registered to the branched anatomical structure which the medical device is moving through, the medical system further comprising:

an insertion sensor for determining an insertion distance of the distal end of the medical device into the branched anatomical structure.

wherein the processor is programmed to identify the set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets by using insertion information from the insertion sensor.

26. A method for determining a position of the medical device in the branched anatomical structure, the method comprising:

determining a most likely match between information that has been extracted from a sequence of images that have been captured from a perspective of a distal end of a medical device as the medical device moves through a path including a plurality of lumens in the branched anatomical structure and corresponding information that has been extracted from the computer model of the branched anatomical structure.

27. The method of claim 26, wherein the sequence of images comprises a plurality of tracklets, wherein each of the plurality of tracklets comprises a subset of the sequence of images which is topologically equivalent, but topologically different than adjacent ones of the plurality of tracklets, and wherein the determining of the most likely match comprises:

identifying a set of potential matches in the computer model of the branched anatomical structure for each of the plurality of tracklets;

determining a probability indicating a quality of match for each member of the set of potential matches for each of the plurality of tracklets; and

determining a maximal combined probability using information of the probabilities for the set of potential matches for each of the plurality of tracklets.

28. The method of claim 27, wherein the determining of the most likely match further comprises:

determining a set of transitional probabilities for each member of the set of potential matches for each of the plurality of tracklets, wherein each member of the set of transitional probabilities indicates a quality of match for a transition from the tracklet to a next in sequence tracklet of the plurality of tracklets; and

determining the maximal combined probability using information of the probabilities and transitional probabilities for the set of potential matches for each of the plurality of tracklets.

29. The method of claim 28, further comprising:

extracting blobs from the sequence of captured images so that each lumen appearing in the sequence of images is indicated by one of the extracted blobs and the information of lumens appearing in the sequence of images comprises information of the extracted blobs.