Image guided surgery with dynamic image reconstruction

• US 9,076,203 B2
• Filed: 11/26/2007
• Issued: 07/07/2015
• Est. Priority Date: 11/26/2007
• Status: Expired due to Fees

• Current Assignee: Gearbox LLC (Intellectual Ventures LLC)
• Original Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Inventors: Myhrvold, Nathan P., Rivet, Dennis J., Smith, Michael A., Weaver, Thomas A., Wood, Lowell L. Jr., Wood, Victoria Y. H., Boyden, Edward S., Hyde, Roderick A., Ishikawa, Muriel Y., Leuthardt, Eric C.
• Primary Examiner(s): Logie, Michael
• Assistant Examiner(s): MCCORMACK, JASON L
• Application Number:
  US11/986,785
• Publication Number:
  US 20090136099A1
• Time in Patent Office: 2,780 Days
• Field of Search: 382/129, 382/154, 345/419, 356/396, 600/414, 600/426
• US Class Current: 1/1
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/30016 Brain
• CPC Class Codes: G06T 2207/30048 Heart; Cardiac
• CPC Class Codes: G06T 2207/30061 Lung
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/147 using affine transformations
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images

Abstract

A method may include processing two or more fiducials included in a three-dimensional medical image and included in a current image to generate two or more transform coefficients of a transform, and applying the transform to the three-dimensional medical image to form a present image.

Claims

1. A system, comprising:

circuitry configured for obtaining a predictive transform based at least partially on (i) structural modeling of a brain, (ii) cyclical stress and strain information associated with at least one observation at a first side of the brain and at least one observation at a substantially opposite side of the brain, and (iii) at least one input related to one or more retractors located in the brain, the circuitry configured for obtaining a predictive transform including at least;

circuitry configured for processing two or more fiducials included in a three-dimensional image of the brain and included in a current image to generate two or more transform coefficients of the predictive transform,

wherein at least one transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a first side of the brain and at least another transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a substantially opposite side of the brain, and

wherein at least one fiducial location in the current image differs from a corresponding fiducial location in the three-dimensional medical image;

circuitry configured for applying the predictive transform to the three-dimensional image to form a present image; and

circuitry configured for forming a time series of images corresponding to two or more instances of the present image, including at least one of extrapolating the time series of images forward to a future time, extrapolating the time series of images backward to a previous time, or interpolating the time series of images to an intermediate time.

2. A system, comprising:

circuitry configured for obtaining a predictive transform associated with an object, the object including one or more of at least some cells, at least some tissue, or at least one organ, based at least partially on (i) structural modeling of the object, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the object and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the object, the circuitry configured for obtaining a predictive transform including at least;

circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform,

wherein at least one transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a reference point with respect to the object and at least another transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a distal point; and

circuitry configured for applying the predictive transform to the three-dimensional image to form a present image.

3. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional image blood vessel locations included in the three-dimensional image of the object and processing two or more current image blood vessel locations included in the current image to generate the two or more transform coefficients.

4. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional image post locations included in the three-dimensional image of the object and processing two or more current image post locations included in the current image to generate the two or more transform coefficients.

5. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional image dot images, formed from a biodegradable marker included in the three-dimensional image of the object, and processing two or more current image dot images, formed from the biodegradable marker included in the current image to generate the two or more transform coefficients.

6. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional image dot images, formed from a contrast enhancing material, in a three-dimensional image including at least the object and processing two or more current image dot images, formed from the contrast enhancing material, in a current image including at least the object to generate the two or more transform coefficients.

7. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional image fiducial locations formed to substantially outline an anatomical feature included in the three-dimensional image of the object and processing two or more current image fiducial locations formed to substantially outline the anatomical feature included in the current image to generate the two or more transform coefficients.

8. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional fiducial locations substantially defined by two or more enhanced fluorescence bead images included in the three-dimensional image of the object and processing two or more current image fiducial locations substantially defined by the two or more enhanced fluorescence bead images included in the current image to generate the two or more transform coefficients.

9. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional fiducial locations substantially defined by two or more tantalum compound bead images included in the three-dimensional image of the object and processing two or more current image fiducial locations substantially defined by the two or more tantalum compound bead images included in the current image to generate the two or more transform coefficients.

10. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more three-dimensional fiducial locations substantially defined by two or more enhanced x-ray marker images included in the three-dimensional image of the object and processing two or more current image fiducial locations substantially defined by the two or more enhanced x-ray marker images included in the current image to generate the two or more transform coefficients.

11. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform including at least an affine transform to the three-dimensional image to form the present image.

12. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform including at least a volume harmonic transform to the three-dimensional image to form the present image.

13. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform including at least an elasticity and modulus transform that includes elasticity and modulus information to form the present image.

14. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform including at least substantial minimization of a function of two or more fiducial locations in the three dimensional image and two or more fiducial locations in the current image.

15. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform including at least a transform including stress and strain information obtained from observing how an elastic wave moves through the current image to form the present image.

16. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform including at least information related to the one or more locations of the one or more medical instruments in the current image to the three-dimensional image to form the present image.

17. The system of claim 2, further comprising:

circuitry configured for forming a time series of images corresponding to two or more instances of the present image including at least one of extrapolating the time series of images forward to a future time, extrapolating the time series of images backward to a previous time, or interpolating the time series of images to an intermediate time.

18. The system of claim 17, wherein circuitry configured for forming a time series of images corresponding to two or more instances of the present image including at least one of extrapolating the time series of images forward to a future time, extrapolating the time series of images backward to a previous time, or interpolating the time series of images to an intermediate time comprises:

circuitry configured for forming a time series of images wherein at least some of the time series of images depict an effect of one or more forces that strain or deform an organ including at least gravity.

19. The system of claim 17, wherein circuitry configured for forming a time series of images corresponding to two or more instances of the present image including at least one of extrapolating the time series of images forward to a future time, extrapolating the time series of images backward to a previous time, or interpolating the time series of images to an intermediate time comprises:

circuitry configured for forming a time series of images wherein at least some of the time series of images depict an effect of one or more forces that strain or deform an organ including at least dynamics.

20. The system of claim 17, wherein circuitry configured for forming a time series of images corresponding to two or more instances of the present image including at least one of extrapolating the time series of images forward to a future time, extrapolating the time series of images backward to a previous time, or interpolating the time series of images to an intermediate time comprises:

circuitry configured for interpolating the time series of images to an intermediate time to form an intermediate time image.

21. The system of claim 17, wherein circuitry configured for forming a time series of images corresponding to two or more instances of the present image including at least one of extrapolating the time series of images forward to a future time, extrapolating the time series of images backward to a previous time, or interpolating the time series of images to an intermediate time comprises:

circuitry configured for extrapolating the time series of images backward to a previous time to form a previous time image.

22. The system of claim 2, further comprising:

circuitry configured for issuing a command to display the present image.

25. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for selecting the three-dimensional medical image from a volume in three-dimensional space.

28. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more numerical values suitable for use in connection with the transform.

29. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing two or more fiducials included in a three-dimensional image and included in a current image to generate two or more complex numerical values suitable for use in connection with the transform, at least some of the complex numerical values having at least a real value and an imaginary value.

30. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform to a first three-dimensional image to form a second three-dimensional image.

31. The system of claim 2, wherein circuitry configured for processing two or more fiducials included in a three-dimensional image of the object and included in a current image to generate two or more transform coefficients of the predictive transform comprises:

circuitry configured for processing fiducial locations selected to substantially outline at least one anatomical feature, wherein the size of the at least one anatomical feature included in the three-dimensional image differs from the size of the at least one anatomical feature in the current image.

32. The system of claim 17, wherein circuitry configured for forming a time-series of images corresponding to two or more instances of the present image including at least one of extrapolating the time series of images forward to a future time, extrapolating the time-series of images backward to a previous time, or interpolating the time series of images to an intermediate time comprises:

circuitry configured for extrapolating the time series of images forward to a future time to form a future time image.

33. The system of claim 2, wherein circuitry configured for applying the predictive transform to the three-dimensional image to form a present image comprises:

circuitry configured for applying the predictive transform including at least applying one or more separate transforms to one or more separate portions of the three-dimensional image to form the present image.

34. The system of claim 33, wherein circuitry configured for applying the predictive transform including at least applying one or more separate transforms to one or more separate portions of the three-dimensional image to form the present image comprises:

circuitry configured for applying the predictive transform including at least applying a first type of transform to at least a first portion of the three-dimensional image and applying a second type of transform to at least a second portion of the three-dimensional image to form the present image.

35. The system of claim 2, wherein circuitry configured for obtaining a predictive transform associated with an object, the object including one or more of at least some cells, at least some tissue, or at least one organ, based at least partially on (i) structural modeling of the object, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the object and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the object comprises:

circuitry configured for obtaining a predictive transform based at least partially on at least one input related to one or more medical instruments located in the object, the one or more medical instruments associated with location information in a coordinate system that at least includes a retractor having coordinates.

36. The system of claim 2, wherein circuitry configured for obtaining a predictive transform associated with an object, the object including one or more of at least some cells, at least some tissue, or at least one organ, based at least partially on (i) structural modeling of the object, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the object and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the object comprises:

circuitry configured for obtaining a mathematical predictive transform based at least partially on (i) structural modeling of the object, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the object and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the object.

37. The system of claim 2, wherein circuitry configured for obtaining a predictive transform associated with an object, the object including one or more of at least some cells, at least some tissue, or at least one organ, based at least partially on (i) structural modeling of the object, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the object and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the object comprises:

circuitry configured for obtaining a predictive affine transform based at least partially on (i) structural modeling of the object, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the object and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the object.

38. The system of claim 2, wherein circuitry configured for obtaining a predictive transform associated with an object, the object including one or more of at least some cells, at least some tissue, or at least one organ, based at least partially on (i) structural modeling of the object, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the object and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the object comprises:

circuitry configured for obtaining a predictive transform operable to account for an effect of one or more natural brain pulses in the transform.

23. A system comprising:

circuitry configured for obtaining a predictive transform based at least partially on (i) structural modeling of a brain, (ii) cyclical stress and strain information associated with at least one observation at a first side of the brain and at least one observation at an opposite side of the brain, and (iii) at least one input related to one or more retractor blades located in the brain, the circuitry configured for obtaining a predictive transform including at least;

circuitry configured for processing two or more fiducials included in a three-dimensional image of the brain and included in a current image to generate two or more transform coefficients of the predictive transform,

wherein at least one transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a first side of the brain and at least another transform coefficient of the two or more transform coefficients is at least partially derived from at least one observation at an opposite side of the brain, and wherein at least one fiducial location in the current image differs from a corresponding fiducial location in the three-dimensional medical image;

circuitry configured for applying the predictive transform to the three-dimensional image to form a present image; and

circuitry configured for forming a time series of images corresponding to two or more instances of the present image, including at least one of extrapolating the time series of images forward to a future time, extrapolating the time series of images backward to a previous time, or interpolating the time series of images to an intermediate time.

24. A system, comprising:

circuitry configured for obtaining a predictive transform based at least partially on (i) structural modeling of a brain, (ii) cyclical stress and strain information associated with at least one observation at a first side of the brain and at least one observation at an opposite side of the brain, and (iii) at least one input related to one or more retractor blades located in the brain, including at least;

processing two or more fiducials included in a three-dimensional image of the brain and included in a current image to generate two or more transform coefficients of the predictive transform,

wherein at least one transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a first side of the brain and at least another transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at an opposite side of the brain, and

wherein at least one fiducial location in the current image differs from a corresponding fiducial location in the three-dimensional image; and

apply the predictive transform to the three-dimensional image to form a present image.

26. A system, comprising:

circuitry configured for obtaining a predictive transform based at least partially on (i) structural modeling of a brain, (ii) cyclical stress and strain information associated with at least one observation at a first side of the brain and at least one observation at a substantially opposite side of the brain, and (iii) at least one input related to one or more medical instruments located in the brain, the circuitry configured for obtaining a predictive transform including at least;

circuitry configured for processing two or more fiducials included in a three-dimensional image of the brain and included in a current image to generate two or more transform coefficients of the predictive transform,

wherein at least one transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a first side of the brain and at least another transform coefficient of the two or more transform coefficients is at least partially derived from at least one observation at an opposite side of the brain; and

circuitry configured for applying the predictive transform to the three-dimensional image to form a present image.

27. A system, comprising:

circuitry configured to obtain for obtaining a predictive transform based at least partially on (i) structural modeling of a brain, (ii) cyclical stress and strain information associated with at least one observation at a reference point with respect to the brain and at least one observation at a distal point, and (iii) at least one input related to one or more medical instruments located in the brain, the circuitry configured for obtaining a predictive transform including at least;

circuitry configured for processing two or more fiducials included in a three-dimensional image of the brain and included in a current image to generate two or more transform coefficients of the predictive transform,

wherein at least one transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a reference point with respect to the brain and at least another transform coefficient of the two or more transform coefficients is at least partially derived from the at least one observation at a distal point; and

circuitry configured for applying the predictive transform to the three-dimensional medical image to form a present image.