Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery

• US 8,900,131 B2
• Filed: 05/13/2011
• Issued: 12/02/2014
• Est. Priority Date: 05/13/2011
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Chopra, Prashant, Donhowe, Caitlin Q., Duindam, Vincent, Prisco, Giuseppe Maria
• Primary Examiner(s): Leubecker, John P
• Assistant Examiner(s): Torres Diaz, Arnaldo
• Application Number:
  US13/107,562
• Publication Number:
  US 20120289777A1
• Time in Patent Office: 1,299 Days
• Field of Search: 600/407, 600424-425
• US Class Current: 600/117
• CPC Class Codes: A61B 1/00055 for alerting the user
• CPC Class Codes: A61B 1/0051 with controlled bending of insertion part
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2017/00323 Cables or rods
• CPC Class Codes: A61B 2017/00699 correcting for movement caused by respiration, e.g. by triggering
• CPC Class Codes: A61B 2017/00703 correcting for movement of heart, e.g. ECG-triggered
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 2576/023 for the heart
• CPC Class Codes: A61B 2576/026 for the brain
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/0037 Performing a preliminary scan, e.g. a prescan for identifying a region of interest
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/02007 Evaluating blood vessel condition, e.g. elasticity, compliance
• CPC Class Codes: A61B 5/02028 Determining haemodynamic parameters not otherwise provided for, e.g. cardiac contractility or left ventricular ejection fraction
• CPC Class Codes: A61B 5/08 Detecting, measuring or recording devices for evaluating the respiratory organs A61B5/0205 takes precedence
• CPC Class Codes: A61B 5/20 for measuring urological functions restricted to the evaluation of the urinary system A61B5/4375 takes precedence
• CPC Class Codes: A61B 5/4064 Evaluating the brain A61B5/031, A61B5/369, A61B5/14553 take precedence
• CPC Class Codes: A61B 5/42 : Detecting, measuring or recording for evaluating the gastrointestinal, the endocrine or the exocrine systems A61B1/00, A61B5/4857, A61B5/486, A61B5/4866, A61B5/4869 take precedence
• CPC Class Codes: A61B 5/6852 : Catheters
• CPC Class Codes: A61B 5/7475 : User input or interface means, e.g. keyboard, pointing device, joystick
• CPC Class Codes: G06T 19/003 : Navigation within 3D models or images
• CPC Class Codes: G06T 2207/10068 : Endoscopic image
• CPC Class Codes: G06T 7/75 : involving models

Abstract

A medical system provides navigation assistance to a surgeon so that the surgeon may navigate a flexible medical device through linked passages of an anatomical structure to a target in or adjacent to the anatomical structure. As the medical device moves through the linked passages, images are captured by an image capturing element at its distal end and pose and shape information for the medical device are received from sensors disposed in the medical device. A 4-D computer model of the anatomical structure is registered to the medical device using one or both of 4-D shape registration and virtual camera registration so that the captured image and a virtual image generated from the perspective of a virtual camera are registered to each other and displayed while providing an indication of a navigational path to the target.

Claims

1. A medical system comprising:

a memory storing information of a four-dimensional computer model of an anatomical structure of a patient, wherein the four-dimensional computer model includes information of a plurality of three-dimensional computer models of the anatomical structure of the patient, wherein each of the plurality of three-dimensional computer models is for a different point in time than others of the plurality of three-dimensional computer models so that the fourth dimension is time, wherein the plurality of three-dimensional computer models indicate movement of the anatomical structure, wherein each of the plurality of three-dimensional computer models defines a plurality of passages, and wherein each of the plurality of passages has a shape;

a medical device having a flexible body and a plurality of sensors distributed along the length of the flexible body; and

a processor programmed to determine the pose and shape of the flexible body while disposed in a passage of the anatomical structure using information provided by the plurality of sensors at the same point in time and register the medical device to the four-dimensional computer model of the anatomical structure by matching at least the determined shape of the flexible body to a best fitting one of the shapes of one or more potential passages in the four-dimensional computer model of the anatomical structure.

2. The medical system of claim 1, wherein the memory stores information of a navigational path relative to the four-dimensional computer model to a target in the anatomical structure, and wherein the processor is programmed to provide an indication of the navigational path as a steering direction for the medical device after registering the medical device to the four-dimensional computer model.

3. The medical system of claim 2, further comprising;

a display;

wherein the processor is programmed to generate a virtual view within the four-dimensional computer model of an interior of the anatomical structure from a perspective of a distal end of the medical device, and wherein the processor is programmed to display the generated virtual view and an indication of the navigational path on the display.

4. The medical system of claim 3, further comprising:

an image capturing device for capturing images from the perspective of the distal end of the medical device;

wherein the processor is programmed to display the captured images on the display.

5. The medical system of claim 2, further comprising:

a virtual reality environment;

wherein the processor is programmed to generate a virtual view within the virtual reality environment of an interior of the anatomical structure from a perspective of a distal end of the medical device, and wherein the processor is programmed to provide one or more sensory indications of the generated virtual view along with one or more sensory indications of the navigational path.

6. The medical system of claim 5, wherein the one or more sensory indications of the navigational path include one or more of the five senses of hearing, sight, touch, smell, and taste.

7. The medical system of claim 1, wherein the plurality of sensors is a plurality of strain sensors.

8. The medical system of claim 7, wherein the plurality of sensors include sensors employing Rayleigh scattering.

9. The medical system of claim 7, wherein the medical device has one or more optical fibers extending through the flexible body and the plurality of sensors include sensors employing fiber Bragg gratings on the one or more optical fibers.

10. The medical system of claim 1, wherein the anatomical structure is a lung and the one or more potential passages include a trachea, bronchi, and bronchioles.

11. The medical system of claim 1, wherein the anatomical structure is one of a heart, brain, digestive system, circulatory system, and urinary system.

12. The medical system of claim 1, further comprising:

an input device;

one or more actuators coupled to the medical device;

wherein the processor is programmed to command the one or more actuators to move the medical device in response to operator movement of the input device.

13. The medical system of claim 1, wherein the medical device includes one of an endoscope, a catheter, and a medical instrument.

14. The medical system of claim 1, wherein each of the plurality of three-dimensional computer models of the anatomical structure is for a different point in time during a periodic motion of the anatomical structure which includes a circulation within a body of at least one of air and blood.

15. The medical system of claim 1, wherein each of the plurality of three-dimensional computer models of the anatomical structure is for a different point in time during a non-periodic motion of the anatomical structure resulting from a body reaction to a stimulus.

16. A method for registering a flexible medical device disposed within a passage in an anatomical structure of a patient to a four-dimensional computer model of the anatomical structure, wherein the flexible medical device has a plurality of sensors distributed along the length of the flexible medical device, wherein the four-dimensional computer model includes information of a plurality of three-dimensional computer models of the anatomical structure, wherein each of the plurality of three-dimensional computer models is for a different point in time than others of the plurality of three-dimensional computer models so that the fourth dimension is time, wherein the plurality of three-dimensional computer models indicate movement of the anatomical structure, wherein each of the plurality of three-dimensional computer models defines a plurality of passages, and wherein each of the plurality of passages has a shape, the method comprising:

determining a current pose and shape of the flexible medical device using information provided by the plurality of sensors at the same point in time;

registering the flexible medical device to the four-dimensional computer model of the anatomical structure by matching at least the determined shape of the flexible medical device to a best fitting one of the shapes of potential passages in the four-dimensional computer model.

17. A medical system comprising:

a memory storing information of a computer model of an anatomical structure;

a medical device;

an image capturing device for capturing images from a perspective of a distal end of the medical device; and

a processor programmed to periodically perform a global registration of the computer model to the medical device by determining the pose and shape of the medical device while disposed in a passage of the anatomical structure and by matching at least the determined shape of the medical device to a best fitting one of the shapes of one or more potential passages in the computer model of the anatomical structure, followed by performing a local registration of the computer model to the medical device by comparing an image captured by the image capturing device with a plurality of virtual views of the computer model of the anatomical structure, wherein the plurality of virtual views is generated from the perspective of a virtual camera whose pose is initially set at the pose of the distal end of the medical device and then perturbed about the initial pose.

18. The medical system of claim 17, further comprising:

a display;

wherein the memory stores information of a navigational path to a target in the anatomical structure, wherein the processor is programmed to generate a corrected virtual view of the anatomical structure using the periodically performed global and local registration of the computer model of the anatomical structure, and wherein the processor is programmed to cause the captured image, the corrected virtual view, and an indication of the navigational path to be displayed on the display.

19. The medical system of claim 18, wherein the processor is programmed to cause the captured image and the corrected virtual view to be displayed on the display in adjacent windows, and wherein the processor is programmed to cause the indication of the navigational path to be displayed on the display as an arrow indicating a direction to be taken in the corrected virtual view.

20. The medical system of claim 18, further comprising:

an input device;

one or more actuators coupled to the medical device;

wherein the processor is programmed to command the one or more actuators to move the medical device in response to operator movement of the input device.

21. The medical system of claim 17, wherein the medical device includes one of an endoscope, a catheter, and a medical instrument.

22. The medical system of claim 17, wherein the processor is programmed to compensate for motion of the anatomical structure by identifying the motion and employing a trigger associated with a point in the identified motion.

23. The medical system of claim 22, wherein the motion is one or a combination of a periodic motion and non-periodic motion associated with the anatomical structure.

24. The medical system of claim 23, wherein the periodic motion includes a circulation within a body of at least one of air and blood.

25. The medical system of claim 23, wherein the non-periodic motion results from a body reaction to a stimulus.

26. The medical system of claim 17, wherein the memory stores information of a plurality of computer models of the anatomical structure, wherein each of the plurality of computer models corresponds to a different point in time over a period of motion of the anatomical structure, wherein the processor is programmed to register the computer model to the medical device by matching at least the determined shape of the flexible body to a best fitting one of shapes of potential passages in the plurality of computer models, and wherein the computer model used by the processor for generating the plurality of virtual views is the computer model corresponding to the matched passage.

27. The medical system of claim 26, wherein the period of motion is associated with at least one of an identifiable periodic motion and identifiable non-periodic motion.

28. The medical system of claim 27, wherein the identifiable periodic motion includes a circulation within a body of at least one of air and blood.

29. The medical system of claim 27, wherein the identifiable non-periodic motion results from a body reaction to a stimulus.

30. A method for registering a computer model of anatomical structure to a medical device, the method comprising:

periodically performing a global registration of the computer model to the medical device by determining the pose and shape of the medical device while disposed in a passage of the anatomical structure and by matching at least the determined shape of the medical device to a best fitting one of the shapes of one or more potential passages in the computer model of the anatomical structure, followed by performing a local registration of the computer model to the medical device by comparing an image captured by the image capturing device with a plurality of virtual views of the computer model of the anatomical structure, wherein the plurality of virtual views is generated from the perspective of a virtual camera whose pose is initially set at the pose of the distal end of the medical device and then perturbed about the initial pose.