REGISTRATION OF VIDEO CAMERA WITH MEDICAL IMAGING

• US 20170084036A1
• Filed: 09/21/2015
• Published: 03/23/2017
• Est. Priority Date: 09/21/2015
• Status: Abandoned Application

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Pheiffer, Thomas, Kluckner, Stefan, Kamen, Ali
• Application Number:
  US14/859,540
• Publication Number:
  US 20170084036A1
• Time in Patent Office: Days
• CPC Class Codes: A61B 1/044 for absorption imaging
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

Intraoperative camera data is registered with medical scan data. The same salient features are located in both the medical scan data and the model from the camera data. The features are specifically labeled rather than just being represented by the data. At least an initial rigid registration is performed using the salient features. The coordinate systems of the camera and the medical scan data are aligned without external positions sensors for the intraoperative camera.

Claims

1. A method for registration of a video camera with a preoperative volume, the method comprising:

fitting an atlas labeled with first salient features to the preoperative volume of a patient;

acquiring depth measurements from an endoscope or laparoscope having the video camera and inserted within the patient;

imaging a medical instrument in the patient with the video camera;

receiving indications of second salient features by the medical instrument being positioned relative to the second salient features;

creating a three-dimensional distribution of the depth measurements labeled with the second salient features;

registering the three-dimensional distribution with the preoperative volume using the second salient features of the three-dimensional distribution and the first salient features of the preoperative volume; and

generating an image of the patient from the preoperative volume and a capture from the video camera, the image being based on the registering of the preoperative volume with the three-dimensional distribution.

2. The method of claim 1 wherein fitting comprises fitting, by a processor, the atlas as a statistical atlas non-rigidly with the preoperative volume.

3. The method of claim 1 wherein acquiring the depth measurements comprises acquiring with a time-of-flight sensor.

4. The method of claim 1 wherein acquiring the depth measurements comprises acquiring with the video camera in a stereo view or from a projection of structured light.

5. The method of claim 1 wherein imaging the medical instrument comprises tracking a tip of the medical instrument in video from the video camera and in relation to the depth measurements.

6. The method of claim 1 wherein receiving the indications comprises receiving user input when a tip of the medical instrument is positioned against the second salient features.

7. The method of claim 1 wherein receiving indications comprises receiving indications of points, surface patches, or points and surface patches.

8. The method of claim 1 wherein creating comprises stitching a video stream from the video camera into the three-dimensional distribution with structure from motion or simultaneous localization and mapping.

9. The method of claim 1 wherein the first and second salient features comprise surfaces and wherein registering comprises a rigid, surface-based registration.

10. The method of claim 9 wherein the rigid, surface-base registration comprises common iterative closest point registration.

11. The method of claim 1 further comprising:

performing non-rigid registration after the registering, the non-rigid registration using residual distances from the registering as partial boundary conditions.

12. The method of claim 1 wherein generating the image comprises generating the image as a three-dimensional rendering of the preoperative volume including a model of the medical instrument positioned based on the registering.

13. The method of claim 1 wherein generating the image comprises generating a visual trajectory of the medical instrument in a rendering of the preoperative volume.

14. The method of claim 1 further comprising repeating the imaging, receiving, and registering with additional second salient features until a metric in the registration is satisfied.

15. In a non-transitory computer readable storage medium having stored therein data representing instructions executable by a programmed processor for registration with medical scan data, the storage medium comprising instructions for:

identifying salient features in the medical scan data representing a patient, the medical scan data being from a medical scanner;

identifying the salient features in video images from an intraoperative camera and positioning of a tool within the patient; and

registering coordinates systems of the medical scan data from the medical scanner with the intraoperative camera using the identified salient features.

16. The non-transitory computer readable storage medium of claim 15 wherein the registering is performed with tracking of the tool without a tracking sensor external to the patient.

17. The non-transitory computer readable storage medium of claim 15 wherein identifying in the medical scan data comprises fitting a statistical model to the medical scan data, the statistical model including the salient features, wherein identifying in the video images comprises segmenting the tool in the video images and placing the tool adjacent to the salient features in the patient, and wherein registering comprises rigidly registering the salient features in the medical scan data with the salient features in a three-dimensional model from the video images.

18. A system for registration, the system comprising:

an intraoperative camera operable to captures images from within a patient;

a minimally invasive surgical tool operable to be inserted into the patient;

a memory configured to store data representing labeled anatomy of the patient, the data being from a medical imager; and

a processor configured to locate anatomical positions using the surgical tool represented in the images and to register the images with the data using the labeled anatomy and the anatomical positions.

19. The system of claim 18 wherein the processor is configured to generate a model of the patient from depth measurements for the images, the anatomical positions located relative to the model.

20. The system of claim 19 further comprising a time-of-flight sensor adjacent to the intraoperative camera.