Medical navigation image output comprising virtual primary images and actual secondary images

• US 9,925,017 B2
• Filed: 03/31/2010
• Issued: 03/27/2018
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Brainlab AG
• Original Assignee: Brainlab AG
• Inventors: Schmidt, Robert
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US12/750,742
• Publication Number:
  US 20100295931A1
• Time in Patent Office: 2,918 Days
• Field of Search: 348 65, 348 77, 348 68, 348 73, 348 69, 348 71, 348 76, 348 79, 600109, 600101, 600117, 600118, 600160, 600178, 600112
• CPC Class Codes: A61B 1/041 Capsule endoscopes for imaging
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/0638 providing two or more wavelengths
• CPC Class Codes: A61B 1/07 using light-conductive means, e.g. optical fibres
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/06 Devices, other than using radiation, for detecting or locating foreign bodies for removing same A61B17/50 ; determining position of probes within or on the body of the patient
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: G02B 21/365 Control or image processing arrangements for digital or video microscopes G02B21/361, G02B21/362 take precedence
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes

Abstract

The invention relates to an image output method for medical navigation in which the positional relationship of an instrument with respect to a part of a patient'"'"'s body is ascertained, and in which the instrument and the part of the patient'"'"'s body are displayed in the correct positional relationship on an image output, wherein the display of the part of the patient'"'"'s body is based on the one hand on virtual image data captured by means of a medical imaging method and on the other hand on actual images captured during navigation, wherein the virtual image data is displayed on the image output primarily and as the basis of the image, and the actual images are superimposed on it merely as an addition and secondarily. It also relates to an image generating and image output device for medical navigation.

Claims

1. An image output method for medical navigation in which the positional relationship of a trackable instrument with respect to a part of a patient'"'"'s body is ascertained, and in which the trackable instrument and the part of the patient'"'"'s body are displayed in real time in the correct positional relationship on an image output, the method comprising:

displaying in real time the part of the patient'"'"'s body based on virtual image data captured via a medical imaging method and on actual images captured in real time during navigation

wherein the virtual image data is displayed on the image output as the primary basis of the image and the actual images captured in real time are correlated with and added into the virtual images as the secondary basis of the image so as to augment the virtual images, wherein the actual images are confined to at least one continuous area of the image output that constitutes less than 50% of the overall area of the image output assembled by the virtual image data and the actual images, and wherein capturing the actual images comprises capturing the actual images using a video image capture unit arranged on the trackable instrument.

2. The method according to claim 1, wherein displaying the part of the patient'"'"'s body comprises weighting the virtual image data is more heavily than the actual images.

3. The method according to claim 1, wherein capturing the virtual image data comprises capturing image data before or during navigation using at least one of computed tomography, magnetic resonance tomography, an x-ray recording or fluoroscopic recording, a PET or SPECT recording.

4. The method according to claim 1, wherein capturing the actual images comprises capturing images which can be visually captured.

5. The method according to claim 4, wherein capturing the actual images comprises capturing images which correspond to an actual view of a region of interest.

6. The method according to claim 4, wherein capturing the actual images comprises capturing images by video capture apparatuses or by an object lens said actual images being captured in real time.

7. The method according to claim 1, wherein capturing the actual images comprises using a microscope or an endoscope to capture the actual images.

8. The method according to claim 1, wherein using the video image capture unit comprises using a camera or a camera light recorder as the video image capture unit.

9. The method according to claim 1, further comprising using a medical treatment instrument as the instrument.

10. The method according to claim 9, wherein using the medical treatment instrument as the instrument includes using a pointer, a pair of tweezers, a scalpel or a clamp as the medical instrument.

11. The method according to claim 1, further comprising using an instrument which is equipped for medical navigation as the instrument.

12. The method according to claim 11, wherein using an instrument equipped for medical navigation includes using an instrument which can be positionally detected by means of a tracking system.

13. The method according to claim 11, wherein using an instrument equipped for medical navigation includes using an instrument that includes a tracking reference.

14. The method according to claim 1, further comprising using an instrument which is pre-calibrated for navigation with a navigation system.

15. The method according to claim 1, further comprising using a plurality of instruments each provided with a video image capture unit, wherein the actual image data obtained from the respective plurality of instruments is used to reciprocally observe or monitor the other instruments and/or to produce supplemental images.

16. The method according to claim 1, further comprising:

using the video image capture unit to detect non-visible light wavelengths; and

using the captured non-visible light for image optimization.

17. The method according to claim 1, further comprising using the combination of the virtual image data and the actual images to positionally register the virtual image data.

18. The method according to claim 17, wherein using the combination of the virtual image data and the actual images to positionally register the virtual image data includes using the combination of virtual image data and actual image data for elastic image data registration.

19. The method according to claim 1, further comprising using the combination of the virtual image data and the actual images to update the virtual image data.

20. The method according to claim 1, further comprising weighting and testing for relevance image information used for assembling the image to be output and omitting less important image constituents from the image and/or intensifying or highlighting more important image constituents in the image.

21. A non-transitory computer readable medium comprising computer executable instructions adapted to perform a method in accordance with claim 1.

23. The method according to claim 1, wherein the actual images are confined to the area that constitutes less than 30% of the overall area of the image output.

24. The method according to claim 1, wherein the actual images are confined to the area that constitutes less than 10% of the overall area of the image output.

22. An image generating and image output device for medical navigation in which the positional relationship of a trackable instrument with respect to a part of a patient'"'"'s body is ascertained, comprising:

an image output on which the trackable instrument and the part of the patient'"'"'s body are displayed in the correct positional relationship;

an image processor which generates in real time a display of the part of the patient'"'"'s body on the basis of virtual image data captured via a medical imaging method and on the basis of actual images captured in real time during navigation,

a video image capture unit configured to be arranged on the trackable instrument, said video image capture unit configured to provide the actual images; and

an image display control which displays the virtual image data on the image output as the primary basis of the image, wherein the actual images are correlated with and added into the virtual image data as the secondary basis of the image so as to augment the virtual images, wherein the actual images are confined to at least one continuous area of the image output that constitutes less than 50% of the overall area of the image output assembled by the virtual image data and the actual images.