Imaging device and method

• US 6,351,573 B1
• Filed: 10/07/1996
• Issued: 02/26/2002
• Est. Priority Date: 01/28/1994
• Status: Expired due to Fees

• Current Assignee: Schneider Advanced Technologies, Inc.
• Original Assignee: Schneider Advanced Technologies, Inc.
• Inventors: Schneider, M. Bret
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Do, Anh Hong
• Application Number:
  US08/725,639
• Time in Patent Office: 1,968 Days
• Field of Search: 382/128, 382/130, 382/131, 382/133, 382/154, 382/278, 382/294, 128/630
• US Class Current: 382/294
• CPC Class Codes: A61B 2090/374 NMR or MRI
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 90/10 for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS

Abstract

A method and apparatus for obtaining and displaying in real time an image of an object obtained by one modality such that the image corresponds to a line of view established by another modality. In a preferred embodiment, the method comprises the following steps: obtaining a follow image library of the object via a first imaging modality; providing a lead image library obtained via the second imaging modality; referencing the lead image library to the follow image library; obtaining a lead image of the object in real time via the second imaging modality along a lead-view; comparing the real time lead image to lead images in the lead image library via digital image analysis to identify a follow image line of view corresponding to the real time lead view; transforming the identified follow image to correspond to the scale, rotation and position of the lead image; and displaying the transformed follow image, the comparing, transforming and displaying steps being performed substantially simultaneously with the step of obtaining the lead image in real time.

Claims

1. A method for obtaining and displaying an image of an object being examined, the method comprising the following steps:

obtaining a follow image library of the interior of the object being examined via a three dimensional imaging modality;

obtaining a real time lead image of the object via a two dimensional imaging modality along an external lead view of a user'"'"'s line of sight;

referencing the real time lead image to the follow image library via digital image analysis to identify a follow image line of view corresponding to the user'"'"'s line of sight;

transforming a follow image from the follow image library to correspond to the real time lead image; and

displaying the transformed follow image in the user'"'"'s line of sight, the referencing, transforming and displaying steps being performed substantially in real time after the step of obtaining the real time lead image.

2. The method of claim 1 further comprising slicing the transformed follow image to a selected depth prior to the displaying step.

3. The method of claim 1 wherein the referencing step comprises segmenting the real time lead image and the follow image into first and second sets of subobjects.

4. The method of claim 3 wherein the identifying step further comprises classifying the subobjects.

5. The method of claim 1 wherein all steps are performed in real time.

6. The method of claim 1 wherein the displaying step comprises using a stereoscopic display.

7. A method for obtaining and displaying an image of an object being examined having an exterior surface and an interior, the method comprising the following steps:

obtaining a real time two dimensional lead image of the exterior surface of the object being examined substantially along a user'"'"'s line of sight to the exterior surface;

referencing the lead image to a three dimensional follow image library to identify a follow image line of view corresponding to the line of sight, the follow image library including at least one follow image of the interior of the object;

transforming a follow image from the follow image library to correspond to the real time lead image; and

displaying the transformed follow image in the user'"'"'s line of sight, the referencing, transforming and displaying steps being performed substantially in real time after the step of obtaining the real time lead image.

8. The method of claim 7 wherein the displaying step comprises forming a composite image from the real time lead image and the transformed follow image and displaying the composite image.

9. The method of claim 7 wherein the displaying step comprises displaying the transformed follow image on a see-through display mounted on the user'"'"'s head.

10. The method of claim 7 further comprising, prior to the step of obtaining the real time lead image:

providing a lead image library, and

correlating the lead image library to the follow image library.

11. The method of claim 10 wherein the step of referencing the real time lead image to the follow image library comprises the use of digital image analysis to correlate the real time lead image with a lead image in the lead image library.

12. The method of claim 10 wherein the step of correlating the lead image library to the follow image library comprises the use of digital image analysis.

13. The method of claim 7 further comprising slicing the transformed follow image to a selected depth prior to the displaying step.

14. The method of claim 7 further comprising tracking an instrument and displaying the instrument and the object together.

15. The method of claim 7 further comprising obtaining a follow image library comprising pre-sliced follow images, the referencing and transforming steps comprising selecting a follow image slice from the follow image library.

16. The method of claim 7 further comprising obtaining the follow image library in real time.

17. An imaging apparatus comprising:

a three dimensional follow image library;

a two dimensional lead imager for obtaining lead images along a user'"'"'s line of sight;

a display for displaying images in the user'"'"'s line of sight; and

processing means for interreferencing in real time lead images from the lead imager of an exterior surface of an object being examined along the user'"'"'s line of sight with a follow image from the follow image library of an interior of the object being examined and for displaying the follow image on the display in the user'"'"'s line of sight.

18. The apparatus of claim 17 further comprising depth control for selecting a depth at which a follow image should be sliced.

19. The apparatus of claim 17 further comprising a lead image library, the processing means further comprising means for interreferencing in real time images from the lead imager with a lead image from the lead image library.

20. The apparatus of claim 17 wherein the display comprises a see-through display.

21. The apparatus of claim 17 wherein the lead imager comprises a video camera.

22. The apparatus of claim 21 wherein the video camera comprises a head-mounted video camera.

23. The apparatus of claim 17 wherein the display is a stereoscopic display.

24. The apparatus of claim 17 further comprising a follow imager, the processing means further comprising means for obtaining follow images from the follow imager for the follow image library in real time.

25. A method for obtaining and displaying an image of a first object being examined having an exterior surface and an interior, the method comprising the steps of:

obtaining a follow image of the first object being examined via a three dimensional imaging modality;

obtaining a real time lead image of the exterior surface of the first object via a two dimensional imaging modality substantially along a user'"'"'s line of sight to the exterior surface;

referencing the lead image to the follow image to identify a follow image line of view according to the user'"'"'s line of sight;

transforming the follow image to correspond to the scale, rotation or position indicated by the lead image; and

displaying the transformed follow image in the user'"'"'s line of sight, the referencing, transforming, and displaying steps being performed substantially in real time after the step of obtaining the lead image.

26. A method for obtaining and displaying an image of a first and second object, the first object being examined and the second object being partially within the interior of the first object, the method comprising the steps of:

obtaining a follow image of the first object being examined via a three dimensional imaging modality;

obtaining a real time lead image of the exterior surface of the first and second objects via a two dimensional imaging modality substantially along a user'"'"'s line of sight to the exterior surfaces;

referencing the lead image to the follow image of the first object to identify a follow image line of view according to the user'"'"'s line of sight;

transforming the follow image of the first object to correspond to the scale, rotation or position indicated by the lead image;

modifying the follow image of the first object to include a representation of the second object that is partially within the interior of the first object according to the scale, rotation or position of the second object in the lead image; and

displaying the follow image in the user'"'"'s line of sight, the referencing, transforming, and modifying being performed substantially in real time after the step of obtaining the lead image.

27. A method for obtaining and displaying an image of a first and second object, the first object being examined and the second object being partially within the interior of the first object, the method comprising the steps of:

obtaining a follow image of the first object being examined via a three dimensional imaging modality;

obtaining an image of the second object via a two dimensional imaging modality from a user'"'"'s point of view;

transforming the follow image of the first object to the scale, rotation or position of the first object from the user'"'"'s point of view;

modifying the follow image of the first object to include a representation of the second object that is partially within the interior of the first object according to the scale, rotation or position of the second object from the user'"'"'s point of view; and

displaying the follow image in the user'"'"'s point of view, the transforming and modifying being performed substantially in real time after the step of obtaining the image of the second object.