Systems and methods for navigation based on ordered sensor records

• US 10,039,473 B2
• Filed: 05/13/2013
• Issued: 08/07/2018
• Est. Priority Date: 05/14/2012
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Zhao, Tao, Chopra, Prashant, Duindam, Vincent, Panescu, Dorin, Fan, Lei
• Primary Examiner(s): TURCHEN, ROCHELLE DEANNA
• Application Number:
  US13/892,974
• Publication Number:
  US 20130303892A1
• Time in Patent Office: 1,912 Days
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 5/065 Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30

Abstract

A method of tracking a medical instrument comprises receiving a model of an anatomical passageway formation and receiving a set of ordered sensor records for the medical instrument. The set of ordered sensor records provide a path history of the medical instrument. The method further comprises registering the medical instrument with the model of the anatomical passageway formation based on the path history.

Claims

1. A method of tracking a medical instrument in a formation of anatomical passageways, the method comprising:

receiving, by a control system, a model of the anatomical passageways;

identifying, by the control system, one or more centerlines of the modeled anatomical passageways, the centerlines including a set of candidate match points of the model, wherein each of the candidate match points includes three-dimensional coordinates in a model space;

receiving, by the control system and while the medical instrument is positioned within the anatomical passageways, a set of ordered sensor records generated by one or more sensors of the medical instrument, the set providing a path history of the medical instrument in the anatomical passageways, wherein each sensor record of the set includes three-dimensional coordinates in a patient coordinate space;

determining, by the control system, a relationship between a first sensor record of the set of ordered sensor records, a second sensor record of the set of ordered sensor records, a first candidate match point of the set of candidate match points, and a second candidate match point of the set of candidate match points, wherein the relationship includes a comparison of the three-dimensional coordinates of the first and second sensor records in the patient coordinate space with the three-dimensional coordinates of the first and second candidate match points in the model space;

registering, by the control system, the medical instrument with the model of the anatomical passageways based on the path history and the relationship to locate the medical instrument within the model of the anatomical passageways;

displaying a virtual visualization image in a display system, the virtual visualization image being based on the registering of the medical instrument with the model of the anatomical passageway and depicting a rendered view of the model of the anatomical passageways from a perspective of the medical instrument within the model of the anatomical passageways; and

implementing, by the control system, commands to move the medical instrument based on the registering of the medical instrument with the model of the anatomical passageway.

2. The method of claim 1 wherein the one or more sensors of the medical instrument include an electromagnetic sensor system having an electromagnetic sensor and shape sensor system having an optical fiber and the set of ordered sensor records include position information expressed as three-dimensional coordinates, orientation information, and shape information.

3. The method of claim 1 wherein determining the relationship includes determining a transition probability indicative of a likelihood that the first and second candidate match points are a best match to the first and second sensor records, wherein the first and second candidate match points are sequential candidate match points and the first and second sensor records are sequential sensor records.

4. The method of claim 1 further comprising tracking the movement of the medical instrument relative to the model of the anatomical passageways.

5. The method of claim 1 wherein the set of ordered sensor records include a set of temporally ordered sensor records.

6. The method of claim 1 wherein the set of ordered sensor records include a set of spatially ordered sensor records corresponding to more than one branch in the anatomical passageways.

7. The method of claim 1 wherein receiving the model of the anatomical passageways includes receiving the model formed from a set of three-dimensional volumetric images.

8. The method of claim 1 wherein receiving a set of ordered sensor records includes receiving the set from an electromagnetic sensor coupled to the medical instrument.

9. The method of claim 1 wherein the set of ordered sensor records includes temporally ordered pose observations for a tip of the medical instrument.

10. The method of claim 1 wherein the set of ordered sensor records includes temporally ordered shape observations for a portion of the medical instrument.

11. The method of claim 1 further comprising filtering the received set of ordered sensor records.

12. The method of claim 11 wherein filtering the received set of ordered sensor records includes creating a subset of the ordered sensor records that includes only sensor data recorded at a selected phase of anatomical movement.

13. The method of claim 12 wherein the selected phase of anatomical movement is an expiration phase of a breathing cycle of a lung.

14. The method of claim 3 wherein determining the transition probability includes determining a distance between the first and second candidate match points.

15. The method of claim 3 wherein determining the transition probability includes determining a distance between a first sensor record associated with the first candidate match point and a second sensor record associated with the second candidate match point.

16. The method of claim 3 wherein determining the transition probability includes determining whether the first and second candidate match points are in contiguous passageways of the model of the anatomical passageways.

18. The method of claim 1 wherein registering includes comparing position data from at least one of the sensor records in the set of ordered sensor records to at least one of the candidate match points defined within a passageway in the model of the anatomical passageways.

19. The method of claim 1 wherein registering includes comparing orientation vector data from at least one of the sensor records in the set of ordered sensor records to at least one of the candidate match points defined within a passageway in the model of the anatomical passageways.

20. The method of claim 1 wherein displaying the virtual visualization image in the display system comprises displaying an internal virtual view of the anatomical passageways as the virtual visualization image.

17. A method of tracking a medical instrument in a formation of anatomical passageways, the method comprising:

receiving, by a control system, a model of the anatomical passageways;

identifying, by the control system, one or more centerlines of the modeled anatomical passageways, the centerlines including a set of candidate match points of the model, wherein each of the candidate match points includes three-dimensional coordinates in a model space;

receiving, by the control system, a set of ordered sensor records generated by one or more sensors of the medical instrument while disposed in the anatomical passageways, the set providing a path history of the medical instrument in the anatomical passageways, wherein each sensor record of the set includes three-dimensional coordinates in a patient coordinate space;

spatially subsampling the set of ordered sensor records such that no more than one sensor record per unit of spatial resolution is included in a subsampled set of ordered sensor records, the unit of spatial resolution corresponding to a spatial resolution of an imaging technology used in generating the model of the anatomical passageways;

determining, by the control system, a relationship between a first sensor record of the subsampled set of ordered sensor records, a second sensor record of the subsampled set of ordered sensor records, a first candidate match point of the set of candidate match points, and a second candidate match point of the set of candidate match points, wherein the relationship includes a comparison of the three-dimensional coordinates of the first and second sensor records in the patient coordinate space with the three-dimensional coordinates of the first and second candidate match points in the model space;

registering, by the control system, the medical instrument with the model of the anatomical passageways based on the path history and the relationship;

displaying a virtual visualization image in a display system, the virtual visualization image being based on the registering of the medical instrument with the model of the anatomical passageway and depicting a view of the model of the anatomical passageways from a perspective of the medical instrument; and

implementing, by the control system, commands to move the medical instrument based on the registering of the medical instrument with the model of the anatomical passageways.

21. A system for tracking a medical instrument in a formation of anatomical passageways, the system comprising:

a control system including a processor-readable storage medium and a processor configured to execute programmed instructions stored on the storage medium to;

receive a model of the anatomical passageways;

identify one or more centerlines of the modeled anatomical passageways, the centerlines including a set of candidate match points of the model, wherein each of the candidate match points includes three-dimensional coordinates in a model space;

receive a set of ordered sensor records generated by a sensor system of the medical instrument while the medical instrument is positioned within the anatomical passageways, the set of ordered sensor records providing a path history of the medical instrument in the anatomical passageways, wherein each sensor record of the set includes ordered position and orientation information from the sensor system, the position information including three-dimensional coordinates in a patient coordinate space;

determine a relationship between a first sensor record of the set of ordered sensor records, a second sensor record of the set of ordered sensor records, a first candidate match point of the set of candidate match points, and a second candidate match point of the set of candidate match points, wherein the relationship includes a comparison of the three-dimensional coordinates of the first and second sensor records in the patient coordinate space with the three-dimensional coordinates of the first and second candidate match points in the model space; and

register the medical instrument with the model of the anatomical passageways based on the path history and the relationship to locate the medical instrument within the model of the anatomical passageways;

display a virtual visualization image in a display system, the virtual visualization image being based on registering of the medical instrument with the model of the anatomical passageway and depicting a view of the model of the anatomical passageways from a perspective of the medical instrument; and

implement, by the control system, commands to move the medical instrument based on the registering of the medical instrument with the model of the anatomical passageways.

22. The system of claim 21 further comprising the medical instrument equipped with the sensor system.

23. The system of claim 21 wherein the medical instrument includes an elongated flexible body.

24. The system of claim 21 wherein the sensor system includes an electromagnetic sensor.

25. The system of claim 24 wherein the sensor system further includes a fiber optic shape sensor.

26. The system of claim 21 wherein determining the relationship includes determining a transition probability indicative of a likelihood that the first and second candidate match points are a best match to the first and second sensor records, wherein the first and second candidate match points are sequential candidate match points and the first and second sensor records are sequential sensor records.

27. The system of claim 21 wherein the processor is further configured for tracking the movement of the medical instrument relative to the model of the anatomical passageways.

28. The system of claim 21 wherein the set of ordered sensor records include a set of temporally ordered sensor records.

29. The system of claim 21 wherein the set of ordered sensor records include a set of spatially ordered sensor records.

30. The system of claim 21 wherein receiving the model of the anatomical passageways includes receiving the model formed from a set of three-dimensional volumetric images.

31. The system of claim 21 wherein receiving a set of ordered sensor records includes receiving the set from an electromagnetic sensor coupled to the medical instrument.

32. The system of claim 21 wherein the set of ordered sensor records includes temporally ordered pose observations for a tip of the medical instrument.

33. The system of claim 21 wherein the set of ordered sensor records includes temporally ordered shape observations for a portion of the medical instrument.

34. The system of claim 21 wherein registering includes filtering the received set of ordered sensor records.

35. The system of claim 34 wherein filtering the received set of ordered sensor records includes creating a subset of the ordered sensor records that includes only sensor data recorded at a selected phase of anatomical movement.

36. The system of claim 35 wherein the selected phase of anatomical movement is an expiration phase of a breathing cycle of a lung.

37. The system of claim 26 wherein determining the transition probability includes determining a distance between the first and second candidate match points.

38. The system of claim 26 wherein determining the transition probability includes determining a distance between a first sensor record associated with the first candidate match point and a second sensor record associated with the second candidate match point.

39. The system of claim 26 wherein determining the transition probability includes determining whether the first and second candidate match points are in contiguous passageways of the model of the anatomical passageways.

40. The system of claim 21 wherein the processor is further configured for spatially subsampling the set of ordered sensor records such that no more than one sensor record per unit of spatial resolution is included in a subsampled set of ordered sensor records.

41. The system of claim 21 wherein registering includes comparing position data from at least one of the sensor records in the set of ordered sensor records to at least one of the candidate match points defined within a passageway in the model of the anatomical passageways.

42. The system of claim 21 wherein registering includes comparing orientation vector data from at least one of the sensor records in the set of ordered sensor records to at least one of the candidate match points defined within a passageway in the model of the anatomical passageways.

43. The system of claim 21 further comprising a display for displaying an internal virtual view of the anatomical passageways.