Method and system for improving 2D-3D registration convergence

• US 10,758,315 B2
• Filed: 10/10/2016
• Issued: 09/01/2020
• Est. Priority Date: 06/21/2012
• Status: Active Grant

• Current Assignee: Globus Medical, Inc.
• Original Assignee: Globus Medical, Inc.
• Inventors: Johnson, Norbert, Crawford, Neil, Forsyth, Jeffrey, Cheng, Yuan, Mokhtar, Jawad
• Primary Examiner(s): Jacob, Oommen
• Application Number:
  US15/289,537
• Publication Number:
  US 20170020630A1
• Time in Patent Office: 1,422 Days
• Field of Search: 600424, 600407
• CPC Class Codes: A61B 17/17 Guides or aligning means for drills , mills, pins or wires
• CPC Class Codes: A61B 2017/00876 magnetic
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/2072 Reference field transducer attached to an instrument or patient
• CPC Class Codes: A61B 2090/034 abutting on parts of the device itself
• CPC Class Codes: A61B 2090/0811 for the position of a particular part of an instrument with respect to the rest of the instrument, e.g. position of the anvil of a stapling instrument
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 2090/3945 Active visible markers, e.g. light emitting diodes
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 90/11 with guides for needles or instruments, e.g. arcuate slides or ball joints
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: A61B 90/39 Markers, e.g. radio-opaque or breast lesions markers
• CPC Class Codes: A61B 90/96 using barcodes
• CPC Class Codes: A61B 90/98 : using electromagnetic means, e.g. transponders
• CPC Class Codes: G06T 2207/10004 : Still image; Photographic image
• CPC Class Codes: G06T 2207/10012 : Stereo images
• CPC Class Codes: G06T 2207/10072 : Tomographic images
• CPC Class Codes: G06T 2207/10081 : Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 : Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10116 : X-ray image
• CPC Class Codes: G06T 2207/20092 : Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/30052 : Implant; Prosthesis
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/30 : Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

A method for registration of digital medical images is provided. The method includes the step of storing a 3D digital medical image having a 3D anatomical feature and a first coordinate system and storing a 2D digital medical image having a 2D anatomical feature and a second coordinate system. The method further includes the steps of storing a placement of a digital medical object on the 3D digital medical image and the 2D digital medical image and generating a simulated 2D digital medical image from the 3D digital medical image, wherein the simulated 2D digital medical image comprises a simulated 2D anatomical feature corresponding to the 3D anatomical feature. The 2D anatomical feature is compared with the simulated 2D anatomical feature until a match is reached and a registration of the first coordinate system with the second coordinate system based on the match is determined.

Claims

1. A surgical robot system comprising:

a robot having a robot arm and a robot base, wherein the robot base comprises an image registration facility configured to receive a three-dimensional (3D) medical image and a two-dimensional (2D) medical image; and

at least one camera configured to detect a one or more tracking markers in an anatomical coordinate system;

wherein the image registration facility is configured to generate a simulated 2D digital medical image from the 3D medical image and match the simulated 2D digital medical image to the 2D medical image,

wherein the image registration facility initially associates the simulated 2D digital medical image to the 2D digital medical image by corresponding a digital medical object placed on the 3D digital image with the digital medical object placed on the 2D digital medical image,

wherein the image registration facility is further configured to register by;

(i) storing a 3D digital medical image comprising a 3D anatomical feature and a first coordinate system,

(ii) storing a 2D digital medical image comprising a 2D anatomical feature and a second coordinate system,

(iii) storing a placement of the digital medical object on the 3D digital medical image and the 2D digital medical image,

(iv) generating a simulated 2D digital medical image from the 3D digital medical image, wherein the simulated 2D digital medical image comprises a simulated 2D anatomical feature corresponding to the 3D anatomical feature,

(v) comparing the 2D anatomical feature with the simulated 2D anatomical feature until a match is reached, and

(vi) registering the first coordinate system with the second coordinate system based on the match,

wherein the position of the robot arm and a position of the patient, using the one or more tracking markers, in the anatomical coordinate system is represented on a display depicting the first coordinate system,

wherein the robot arm is coupled to an end effector, the end effector being configured with one or more tracking markers on the surface of the end effector to be monitored when the end effector is translated and rotated in a surgical field

wherein the end effector includes a clamp configured to be a ground link and fasten the end effector to the robot arm,

wherein the robot arm includes a ring that is configured as a visual indicator to notify a user of different modes the system is operating under and configured to provide warnings to a user.

2. The system of claim 1, wherein the 3D digital image is generated by at least one of a computed tomography (CT) system and magnetic resonance imaging (MRI) system.

3. The system of claim 1, wherein the 2D digital image is generated by an x-ray system.

4. The system of claim 1, wherein the anatomical coordinate system is at least in part determined by an image tracking system that tracks the location of the patient.

5. The system of claim 1, wherein the 3D digital medical image is pre-operatively obtained from one of a tomography (CT) system and magnetic resonance imaging (MRI) system, and the 2D digital medical image is inter-operatively obtained from an x-ray system.

6. The system of claim 1, wherein the digital medical object is a surgical medical implant device.

7. The system of claim 1, wherein the digital medical object is at least one of a digitally-simulated bio-active implant device, electronics-based medical implant device, and artificial medical transplant material.

8. The system of claim 1, wherein the placement of the digital medical object appears as a shadow.

9. The system of claim 1, wherein the placement of the digital medical object provides a region of interest upon which the step of comparing begins.

10. The system of claim 1, wherein the placement of the corresponding digital medical object placed the 3D digital image and the acquired 2D digital medical image is provided with a placement tolerance.

11. The system of claim 1, wherein the match is determined within a matching tolerance.

12. A method for registration of digital medical image coordinate systems, comprising the steps of:

providing a robotic system having a robot arm and a robot base, wherein the robot base comprises an image registration facility configured to receive a three-dimensional (3D) medical image and a two-dimensional (2D) medical image;

storing a 3D digital medical image comprising a 3D anatomical feature and a first coordinate system;

storing a 2D digital medical image comprising a 2D anatomical feature and a second coordinate system;

storing a placement of a digital medical object on the 3D digital medical image and the 2D digital medical image;

generating a simulated 2D digital medical image from the 3D digital medical image, wherein the simulated 2D digital medical image comprises a simulated 2D anatomical feature corresponding to the 3D anatomical feature;

comparing the 2D anatomical feature with the simulated 2D anatomical feature until a match is reached; and

determining a registration of the first coordinate system with the second coordinate system based on the match

wherein the robot arm is coupled to an end effector, the end effector being configured with one or more tracking markers on the surface of the end effector to be monitored when the end effector is translated and rotated in a surgical field

wherein the end effector includes a clamp configured to be a ground link and fasten the end effector to the robot arm,

wherein the robot arm includes a ring that is configured as a visual indicator to notify a user of different modes the system is operating under and configured to provide warnings to a user.

13. The method of claim 12, wherein the 3D digital image is generated by at least one of a computed tomography (CT) system and magnetic resonance imaging (MRI) system.

14. The method of claim 12, wherein the 2D digital image is generated by an x-ray system and the second coordinate system is in reference to an anatomical coordinate system of a patient during a medical procedure.

15. The method of claim 14, wherein the anatomical coordinate system is at least in part determined by an image tracking system that tracks the location of the patient and the x-ray system.

16. The method of claim 12, wherein the 3D digital medical image is pre-operatively obtained from one of a tomography (CT) system and magnetic resonance imaging (MM) system, and the 2D digital medical image is inter-operatively obtained from an x-ray system.

17. The method of claim 12, wherein the digital medical object is a digitally-simulated surgical medical implant device.

18. The method of claim 12, wherein the digital medical object is at least one of a digitally-simulated bio-active implant device, electronics-based medical implant device, and artificial medical transplant material.

19. The method of claim 12, wherein the placement of the digital medical object appears as a replica of a real object on a medical image.

20. The method of claim 12, wherein the placement of the corresponding digital medical object placed the 3D digital image and the acquired 2D digital medical image is provided with a placement tolerance.