System and method for non-invasive patient-image registration

• US 9,554,117 B2
• Filed: 11/12/2013
• Issued: 01/24/2017
• Est. Priority Date: 08/13/2013
• Status: Active Grant

• Current Assignee: Korea Advanced Institute of Science and Technology
• Original Assignee: Korea Advanced Institute of Science and Technology
• Inventors: Lim, Sunghwan, Lee, Deukhee, Park, Se Hyung
• Primary Examiner(s): GEROLEO, FRANCIS
• Application Number:
  US14/077,462
• Publication Number:
  US 20150049174A1
• Time in Patent Office: 1,169 Days
• Field of Search: 348/50
• US Class Current: 1/1
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 6/504 for diagnosis of blood vessels, e.g. by angiography
• CPC Class Codes: A61B 8/0891 for diagnosis of blood vessels
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/20044 Skeletonization; Medial axis transform
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 2207/30172 Centreline of tubular or elongated structure
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

A system for non-invasive registration between a patient and a three-dimensional (3D) medical image includes a near infrared 3D camera 110 which extracts a 3D blood vessel image I2 of a patients registration target area during surgical operation; a camera position tracer 120 which traces a position of the near infrared 3D camera 110 and calculates a real world coordinate system of the 3D blood vessel image I2; a controller 130 which extracts a first blood vessel pattern from a 3D medical image I1 of the registration target area, extracts a second blood vessel pattern from the 3D blood vessel image I2, and performs position registration between the patient and the 3D medical image I1 through the extracted first and second blood vessel patterns; and a display 140 which displays a registration result calculated by the controller 130.

Claims

1. A system for non-invasive registration between a patient and a three-dimensional (3D) medical image, the system comprising:

a near infrared 3D camera configured to extract a 3D blood vessel image of a patient'"'"'s registration target area;

a camera position tracer configured to trace a position of the near infrared 3D camera and to calculate a real world coordinate system of the 3D blood vessel image;

a controller comprising;

a 3D medical image database configured to store a 3D medical image obtained by taking the registration target area;

a blood vessel model generator configured to reconstitute a blood vessel model by separating a blood vessel image from the 3D medical image;

a first feature point extractor configured to extract a branch point from which a blood vessel is branched by detecting a skeleton line of the blood vessel model, and to select each extracted branch point as a feature point of a first blood vessel pattern;

a 3D blood vessel image detector configured to extract a 3D blood vessel image by separating a blood vessel image from a near infrared 3D image taken by the near infrared 3D camera, and a second feature point extractor configured to extract a branch point of a patient'"'"'s blood vessel through the extracted 3D blood vessel image and a real world coordinate system of the 3D blood vessel image measured by the camera position tracer, and to select each extracted branch point as a feature point of a second blood vessel pattern; and

a registrator configured to determine a registration result by optimally matching a first feature point group comprising of a plurality of first feature points, and a second feature point group comprising of a plurality of second feature points; and

a display configured to display the registration result.

2. The system according to claim 1, wherein

the first feature point extractor further configured to store 3D coordinates of a feature point of a first blood vessel pattern selected with respect to a medical image coordinate system,

the near infrared 3D camera is further configured to take blood vessel images seen by near infrared light through the patient'"'"'s registration target area to obtain two 3D blood vessel images spaced apart at a distance to have parallax, and

the second feature point extractor is further configure to select a second feature point represented by 3D coordinates with respect to a 3D camera coordinate system, using parallax of a pair of corresponding branch points respectively calculated from two 3D blood vessel images of the near infrared 3D camera.

3. The system according to claim 2, wherein

the camera position tracer is further configured to measure six degrees of freedom in position of the near infrared 3D camera, which comprises three rotation positions and three translation positions, and to calculate a coordinate transformation matrix from the position tracer coordinate system to the 3D camera coordinate system by tracing the position of the near infrared 3D camera in real time,

the second feature point extractor is further configured to transform and to save the second feature point in the 3D camera coordinate system into a position tracer coordinate system by the coordinate transformation matrix, and

the registrator is a non-invasive registration system between patient and 3D medical image, and is further configured to calculate a position relationship of a position tracer coordinate system with respect to a medical image coordinate system in a state that a second feature point group in the position tracer coordinate system is rotated and translated to be optimally matched with the first feature point group.

4. The system according to claim 3, wherein the camera position tracer comprises a mechanical position tracing device which is fastened to an end of an articulated link and is configured to measure six degrees of freedom in position of the near infrared 3D camera by sensing a physical displacement due to link movement of joints.

5. The system according to claim 3, wherein the camera position tracer comprises an optical position tracing device which is configured to use a 3D camera taking fiducial markers mounted to the near infrared 3D camera to calculate a 3D position vector and a 3D direction vector of the fiducial marker and to measure six degrees of freedom in position of the near infrared 3D camera.

6. The system according to claim 1, wherein the first feature point extractor is further configured to extract a border line representing an outline of a blood vessel from the blood vessel model, and to extract a center line, wherein the center line is extracted from the extracted border line along the blood vessel, as the skeleton line.

7. A method of non-invasive registration between a patient and a three-dimensional (3D) medical image, the method comprising:

extracting a 3D blood vessel image by taking a blood vessel image by passing near infrared light through a registration target area with a near infrared 3D camera;

tracing a camera position by tracing a position of the near infrared 3D camera in real time with a camera position tracer and calculating a real world coordinate system of the 3D blood vessel image;

storing a 30 medical image obtained by taking the registration target area;

reconstituting a blood vessel model by separating a blood vessel image from the 3D medical image;

extracting a branch point from which a blood vessel is branched by detecting a skeleton line of the blood vessel model;

selecting each extracted branch point as a feature point of a first blood vessel pattern;

extracting a 3D blood vessel image by separating a blood vessel image from a near infrared 3D image taken by the near infrared 3D camera;

extracting a branch point of a patient'"'"'s blood vessel through the extracted 3D blood vessel image and a real world coordinate system of the 3D blood vessel image measured by the camera position tracer;

selecting each extracted branch point as a feature point of a second blood vessel pattern,

determining a registration result by optimally matching a first feature point group comprising of a plurality of first feature points, and a second feature point group comprising of a plurality of second feature points; and

displaying the registration result on a screen.