Surgical planning system and navigation system

• US 9,439,623 B2
• Filed: 05/22/2012
• Issued: 09/13/2016
• Est. Priority Date: 05/22/2012
• Status: Active Grant

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Covidien LP (Medtronic Plc)
• Inventors: Frank, Kevin J., Case, Jason A., Ladtkow, Casey M.
• Primary Examiner(s): Park, Patricia
• Application Number:
  US13/477,417
• Publication Number:
  US 20130317353A1
• Time in Patent Office: 1,575 Days
• Field of Search: 600/426
• US Class Current: 1/1
• CPC Class Codes: A61B 18/1477 Needle-like probes
• CPC Class Codes: A61B 18/1815 using microwaves
• CPC Class Codes: A61B 2017/00477 Coupling A61B2017/0046 takes precedence
• CPC Class Codes: A61B 2018/00577 Ablation
• CPC Class Codes: A61B 2018/1425 Needle
• CPC Class Codes: A61B 2018/183 characterised by the type of antenna
• CPC Class Codes: A61B 2018/1869 with an instrument interstitially inserted into the body, e.g. needles
• CPC Class Codes: A61B 2034/104 Modelling the effect of the tool, e.g. the effect of an implanted prosthesis or for predicting the effect of ablation or burring
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2063 Acoustic tracking systems, e.g. using ultrasound
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/0463 against cooling or freezing
• CPC Class Codes: A61B 2090/0472 against ultrasound energy
• CPC Class Codes: A61B 2090/0481 against EM radiation, e.g. microwave
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 2090/3925 ultrasonic
• CPC Class Codes: A61B 2090/3983 Reference marker arrangements for use with image guided surgery
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 : Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 : User interfaces for surgical systems
• CPC Class Codes: A61B 8/0841 : for locating instruments
• CPC Class Codes: A61B 8/565 : involving data transmission via a network
• CPC Class Codes: A61B 90/04 : Protection of tissue around surgical sites against effects of non-mechanical surgery, e.g. laser surgery protection of eyes against laser radiation A61F9/022
• CPC Class Codes: G06T 2207/10132 : Ultrasound image
• CPC Class Codes: G06T 2207/30004 : Biomedical image processing
• CPC Class Codes: G06T 2207/30208 : Marker matrix
• CPC Class Codes: G06T 2207/30244 : Camera pose
• CPC Class Codes: G06T 2210/41 : Medical
• CPC Class Codes: G06T 7/75 : involving models

Abstract

The present disclosure is directed to a planning and navigation system. The planning system includes a memory configured to store a plurality of images and a controller configured to render the plurality of images in three dimensions, automatically segment the plurality of images to demarcate a target area, and automatically determine a treatment plan based on the target area. The navigation system includes an ultrasound device having a fiducial pattern that obtains an ultrasound image, a surgical device having a image capture device that captures an image of the fiducial pattern, and a controller that receives the ultrasound image and the fiducial image. The controller determines a position of the surgical device based on the fiducial image. The planning and navigation system also includes a controller that receives data from the planning system and the navigation system and a display configured to display the received information.

Claims

1. A planning and navigation system, comprising:

a planning system including;

a memory configured to store a plurality of images; and

a first controller configured to render the plurality of images in three dimensions, segment the plurality of images to demarcate a target area, and automatically determine a treatment plan based on the target area; and

a navigation system including;

an ultrasound device configured to obtain an ultrasound image in a scan plane, the ultrasound device having a fiducial pattern disposed thereon, the fiducial pattern including a plurality of first unique identifiers disposed in a region and a plurality of second unique identifiers;

a surgical device having an image capture device configured to capture a fiducial image of the fiducial pattern;

a second controller configured to;

receive the ultrasound image and the fiducial image, wherein the controller determines a position of the surgical device in relation to the scan plane based on the fiducial image;

identify the plurality of first unique identifiers and the plurality of second unique identifiers, wherein the plurality of first unique identifiers includes circles of a first color and the plurality of second unique identifiers includes circles of a second color and wherein identifying includes;

applying a first threshold to the fiducial image substantially in parallel with applying a second threshold to an inverted fiducial image;

performing a connected component analysis on the first and second thresholds; and

applying a geometric filter to results of the connected component analysis to identify the region having the plurality of first unique identifiers and to determine weighted centroids of the plurality of first unique identifiers and the plurality of second unique identifiers; and

correspond the fiducial image to a model image by;

selecting a plurality of the circles of the first color;

determining an amount of the circles of the first color disposed in a first portion of the region of the fiducial pattern;

determining whether the determined amount of the circles of the first color disposed in the first portion matches a predetermined amount of the circles of the first color; and

in response to a determination that the determined amount of the circles of the first color disposed in the first portion does not match the predetermined amount of the circles of the first color, determining an amount of the circles of the first color disposed in a second portion of the region of the fiducial pattern; and

a third controller configured to receive the rendered plurality of images, the target area, the treatment plan, the ultrasound image, and the position of the surgical device in relation to the scan plane; and

a display configured to;

display a first display having the rendered plurality of images, the target area, and the treatment plan; and

display a second display having the ultrasound image, a virtual image of the surgical device based on the position of the surgical device in relation to the scan plane, and the treatment plan.

2. The planning and navigation system of claim 1, wherein the first display and the second display are displayed on a single screen.

3. The planning and navigation system of claim 2, wherein the first display and second display are displayed simultaneously.

4. The planning and navigation system of claim 2, wherein a user can switch between the first display and the second display.

5. The planning and navigation system of claim 1, wherein the display has two screens and the first display is displayed on a first screen and the second display is displayed on a second screen.

6. The planning and navigation system of claim 1, wherein the first controller performs a volumetric analysis to determine the treatment plan.

7. The planning and navigation system of claim 1, further comprising an input device configured to adjust the treatment plan.

8. The planning and navigation system of claim 1, wherein the display provides a graphical user interface.

9. The planning and navigation system of claim 1, wherein the first controller segments at least one vessel and adjusts the treatment plan based on the proximity of the at least one vessel to the target area.

10. The planning and navigation system of claim 1, wherein the first controller segments at least one organ and adjusts the treatment plan based on a position of the target area in relation to the at least one organ.

11. The planning and navigation system of claim 1, wherein the fiducial pattern is affixed to a known location on the ultrasound device.

12. The planning and navigation system of claim 1, wherein the image capture device is affixed to a known location on the surgical device.

13. The planning and navigation system of claim 1, wherein the region is of the second color.

14. The planning and navigation system of claim 1, wherein the second controller estimates a camera pose, and transforms the surgical device to model coordinates.

15. The planning and navigation system of claim 1, wherein the selected plurality of the circles of the first color is selected based on roundness of the circles of the first color.

16. The planning and navigation system of claim 1, wherein corresponding the fiducial image to the model image further includes:

sorting the selected plurality of the circles of the first color;

computing a planar homography matrix;

transforming the model image; and

identifying matching image fiducials.