Apparatus and methods for determining a plurality of local calibration factors for an image

• US 9,144,394 B2
• Filed: 12/04/2013
• Issued: 09/29/2015
• Est. Priority Date: 11/18/2008
• Status: Active Grant

• Current Assignee: Sync-Rx, Ltd. (Koninklijke Philips N.V.)
• Original Assignee: Sync-Rx, Ltd. (Koninklijke Philips N.V.)
• Inventors: Cohen, Ran, Klaiman, Eldad, Steinberg, Alexander, Tolkowsky, David
• Primary Examiner(s): Le, Long V
• Assistant Examiner(s): HOFFA, ANGELA MARIE
• Application Number:
  US14/096,968
• Publication Number:
  US 20140094689A1
• Time in Patent Office: 664 Days
• Field of Search: 600/424
• US Class Current: 1/1
• CPC Class Codes: A61B 2090/3966 Radiopaque markers visible in an X-ray image
• CPC Class Codes: A61B 5/0066 Optical coherence imaging
• CPC Class Codes: A61B 5/0073 by tomography, i.e. reconstruction of 3D images from 2D projections A61B5/0066 takes precedence
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/0086 using infrared radiation
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: A61B 6/5217 extracting a diagnostic or physiological parameter from medical diagnostic data
• CPC Class Codes: A61B 8/12 in body cavities or body tracts, e.g. by using catheters catheters per se A61M25/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/10121 Fluoroscopy
• CPC Class Codes: G06T 2207/10132 Ultrasound image
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 2207/30172 Centreline of tubular or elongated structure
• CPC Class Codes: G06T 2207/30204 : Marker
• CPC Class Codes: G06T 7/33 : using feature-based methods

Abstract

Apparatus and methods are described including acquiring a first set of extraluminal images of a lumen, using an extraluminal imaging device. At least one of the first set of images is designated as a roadmap image. While an endoluminal device is being moved through the lumen, a second set of extraluminal images is acquired. A plurality of features that are visible within images belonging to the second set of extraluminal images are identified. In response to the identified features in the images belonging to the second set of extraluminal images, a plurality of local calibration factors associated with respective portions of the roadmap image are determined. Other applications are also described.

Claims

1. An apparatus comprising:

an endoluminal device adapted to be moved through a lumen of a subject'"'"'s body;

one or more extraluminal imaging devices configured to acquire extraluminal two-dimensional (2D) projection images of the lumen;

a display; and

at least one computer processor configured;

to receive from the one or more extraluminal imaging devices;

a first set of one or more extraluminal 2D projection images of the lumen, and

a second set of extraluminal 2D projection images of the endoluminal device inside the lumen, the second set of extraluminal 2D projection images being acquired while the endoluminal device is being moved through the lumen,

to designate at least one of the first set of one or more extraluminal 2D projection images as a roadmap image,

for at least a portion of the images belonging to the second set of extraluminal 2D projection images, to identify within the image a plurality of features that are visible within the image,

in response to the identified features in the images belonging to the second set of extraluminal 2D projection images, to determine a plurality of local calibration factors associated with respective portions of the roadmap image, each of the calibration factors indicating a relationship between a physical dimension of a respective portion of the lumen and a number of pixels in a portion of the roadmap image that corresponds to the physical dimension of the respective portion of the lumen, and

based upon one or more of the determined local calibration factors, to generate an output on the display.

2. The apparatus according to claim 1, wherein the processor is configured to determine the plurality of local calibration factors associated with respective portions of the roadmap image based upon a known speed at which the endoluminal device is moved through the lumen.

3. The apparatus according to claim 1, wherein the processor is configured to determine the plurality of local calibration factors associated with respective portions of the roadmap image by determining local relative calibration factors of the portions of the roadmap image with respect to each other.

4. The apparatus according to claim 1, wherein the processor is configured to determine the plurality of local calibration factors associated with respective portions of the roadmap image based upon a known physical dimension associated with one or more of the identified features.

5. The apparatus according to claim 1,

wherein the endoluminal device comprises a first endoluminal data-acquisition device adapted to acquire a plurality of endoluminal data points while the endoluminal data-acquisition device is being moved through the lumen from a starting location within the lumen;

and

wherein the at least one processor is further configured to co-register respective endoluminal data points to respective locations within the roadmap image, by;

identifying the starting location of the endoluminal data-acquisition device in the roadmap image; and

determining a distance from the starting location at which respective endoluminal data points were acquired, based upon a speed at which the endoluminal data-acquisition device was moved through the lumen, a frame rate at which the endoluminal data points were acquired, and the local calibration factor associated with the respective portions of the roadmap image; and

wherein the processor is configured to generate the output on the display based upon the co-registration of the endoluminal data points to the respective locations within the roadmap image.

6. The apparatus according to claim 5,

further comprising a second endoluminal data-acquisition device adapted to acquire an additional plurality of endoluminal data points while the second endoluminal data-acquisition device is being moved through the lumen,

wherein the processor is configured to co-register respective endoluminal data points of the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective endoluminal data points of the plurality of endoluminal data points acquired by the first endoluminal data-acquisition device by co-registering the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective locations within the roadmap image.

7. The apparatus according to claim 1, wherein the at least one processor is configured, based upon the local calibration factors, to determine locations of the endoluminal device in respective extraluminal 2D projection images of the second set of extraluminal 2D projection images with respect to the roadmap image.

8. The apparatus according to claim 7, wherein the at least one processor is configured, in near real-time with respect to acquisitions of the extraluminal 2D projection images of the second set of extraluminal 2D projection images, to determine locations of the endoluminal device in respective extraluminal 2D projection images of the second set of extraluminal 2D projection images with respect to the roadmap image, and wherein the processor is configured to generate the output by generating an output that is indicative of the determined near-real-time location of the endoluminal device with respect to the roadmap image.

9. The apparatus according to claim 7,

wherein the endoluminal device comprises a first endoluminal data-acquisition device adapted to acquire a plurality of endoluminal data points while the endoluminal data-acquisition device is being moved through the lumen,

wherein the at least one processor is further configured, based upon determining locations of the endoluminal device in respective extraluminal 2D projection images of the second set of extraluminal 2D projection images with respect to the roadmap image, to co-register respective endoluminal data points to respective locations within the roadmap image, and

wherein the processor is configured to generate the output on the display based upon the co-registration of the endoluminal data points to the respective locations within the roadmap image.

10. The apparatus according to claim 9,

further comprising a second endoluminal data-acquisition device adapted to acquire an additional plurality of endoluminal data points while the second endoluminal data-acquisition device is being moved through the lumen,

wherein the processor is configured to co-register respective endoluminal data points of the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective endoluminal data points of the plurality of endoluminal data points acquired by the first endoluminal data-acquisition device by co-registering the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective locations within the roadmap image.

11. The apparatus according to claim 9, wherein the endoluminal data-acquisition device comprises an endoluminal imaging device that is adapted to acquire a plurality of endoluminal images while the endoluminal imaging device is being moved through the lumen, and wherein the processor is configured to co-register respective endoluminal data points to respective locations within the roadmap image by co-registering respective endoluminal images to respective locations within the roadmap image.

12. The apparatus according to claim 9, wherein the endoluminal data-acquisition device comprises an endoluminal data-acquisition device that is adapted to acquire functional data regarding the lumen while the endoluminal data-acquisition device is being moved through the lumen, and wherein the processor is configured to co-register respective endoluminal data points to respective locations within the roadmap image by co-registering respective functional endoluminal data points to respective locations within the roadmap image.

13. The apparatus according to claim 9, wherein the at least one processor is further configured, based upon the co-registration, to generate a stack of endoluminal data points, in which relative dispositions of endoluminal data points within the stack correspond to relative locations of the endoluminal data points with respect to the roadmap image.

14. The apparatus according to claim 13,

wherein the at least one processor is further configured, based upon the co-registering of the endoluminal data points to respective locations within the roadmap image, to determine a parameter of a portion of the lumen that corresponds to a portion of the stack of endoluminal data points, and

wherein the processor is configured to generate the output in response to the determined parameter.

15. The apparatus according to claim 1, wherein the processor is configured to designate the at least one of the first set of one or more extraluminal 2D projection images as the roadmap image, at least partially in response to an input from a user.

16. A method for use with an endoluminal device configured to be moved through a lumen of a subject'"'"'s body, an extraluminal imaging device configured to acquire extraluminal two-dimensional (2D) projection images of the lumen, and a display, the method comprising:

using the extraluminal imaging device, acquiring a first set of one or more extraluminal 2D projection images of the lumen;

using at least one computer processor, designating at least one of the first set of images as a roadmap image;

moving the endoluminal device through the lumen;

while the endoluminal device is being moved through the lumen, acquiring a second set of extraluminal 2D projection images of the endoluminal device inside the lumen, using the extraluminal imaging device;

using the processor, identifying, within each of at least a portion of the images belonging to the second set of extraluminal 2D projection images, a plurality of features that are visible in the image;

using the processor, in response to the identified features in the images belonging to the second set of extraluminal 2D projection images, determining a plurality of local calibration factors associated with respective portions of the roadmap image, each of the calibration factors indicating a relationship between a physical dimension of a respective portion of the lumen and a number of pixels in a portion of the roadmap image that corresponds to the physical dimension of the respective portion of the lumen; and

using the processor, generating an output on the display, based upon one or more of the determined local calibration factors.

17. The method according to claim 16, wherein determining the plurality of local calibration factors associated with respective portions of the roadmap image comprises determining the calibration factors based upon a known speed at which the endoluminal device is moved through the lumen.

18. The method according to claim 16, wherein determining the plurality of local calibration factors associated with respective portions of the roadmap image comprises determining local relative calibration factors of the portions of the roadmap image with respect to each other.

19. The method according to claim 16, wherein determining the plurality of local calibration factors associated with respective portions of the roadmap image comprises determining the calibration factors based upon a known physical dimension associated with one or more of the identified features.

20. The method according to claim 16,

wherein the endoluminal device includes a first endoluminal data-acquisition device configured to acquire a plurality of endoluminal data points while the endoluminal data-acquisition device is being moved through the lumen from a starting location within the lumen,

the method further comprising co-registering respective endoluminal data points to respective locations within the roadmap image, by:

identifying the starting location of the endoluminal data-acquisition device in the roadmap image; and

determining a distance from the starting location at which respective endoluminal data points were acquired, based upon a speed at which the endoluminal data-acquisition device was moved through the lumen, a frame rate at which the endoluminal data points were acquired, and the local calibration factor associated with the respective portions of the roadmap image;

wherein generating the output comprises generating an output on the display based upon the co-registration of the endoluminal data points to the respective locations within the roadmap image.

21. The method according to claim 20,

the method being for use with a second endoluminal data-acquisition device configured to acquire an additional plurality of endoluminal data points while the second endoluminal data-acquisition device is being moved through the lumen,

the method further comprising co-registering respective endoluminal data points of the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective endoluminal data points of the plurality of endoluminal data points acquired by the first endoluminal data-acquisition device, by co-registering the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective locations within the roadmap image.

22. The method according to claim 16, further comprising, based upon the local calibration factors, determining locations of the endoluminal device in respective extraluminal 2D projection images of the second set of extraluminal 2D projection images with respect to the roadmap image.

23. The method according to claim 22, wherein determining locations of the endoluminal device in respective extraluminal 2D projection images of the second set of extraluminal 2D projection images with respect to the roadmap image comprises, in near real-time with respect to acquisitions of the extraluminal 2D projection images of the second set of extraluminal 2D projection images, determining locations of the endoluminal device in respective extraluminal 2D projection images of the second set of extraluminal 2D projection images with respect to the roadmap image, and wherein generating the output comprises generating an output that is indicative of the determined near-real-time location of the endoluminal device with respect to the roadmap image.

24. The method according to claim 22,

wherein the endoluminal device includes a first endoluminal data-acquisition device configured to acquire a plurality of endoluminal data points while the endoluminal data-acquisition device is being moved through the lumen,

the method further comprising, based upon determining locations of the endoluminal device in respective extraluminal 2D projection images of the second set of extraluminal 2D projection images with respect to the roadmap image, co-registering respective endoluminal data points to respective locations within the roadmap image, and

wherein generating the output comprises generating an output on the display based upon the co-registration of the endoluminal data points to the respective locations within the roadmap image.

25. The method according to claim 24,

the method being for use with a second endoluminal data-acquisition device configured to acquire an additional plurality of endoluminal data points while the second endoluminal data-acquisition device is being moved through the lumen,

the method further comprising co-registering respective endoluminal data points of the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective endoluminal data points of the plurality of endoluminal data points acquired by the first endoluminal data-acquisition device by co-registering the additional plurality of endoluminal data points acquired by the second endoluminal data-acquisition device to respective locations within the roadmap image.

26. The method according to claim 24, wherein the endoluminal data-acquisition device includes an endoluminal imaging device that is configured to acquire a plurality of endoluminal images while the endoluminal imaging device is being moved through the lumen, and wherein co-registering respective endoluminal data points to respective locations within the roadmap image comprises co-registering respective endoluminal images to respective locations within the roadmap image.

27. The method according to claim 24, wherein the endoluminal data-acquisition device includes an endoluminal data-acquisition device that is configured to acquire functional data regarding the lumen, while the endoluminal data-acquisition device is being moved through the lumen, and wherein co-registering respective endoluminal data points to respective locations within the roadmap image comprises co-registering respective functional endoluminal data points to respective locations within the roadmap image.

28. The method according to claim 24, wherein generating the output based upon the co-registration comprises generating a stack of endoluminal data points, in which relative dispositions of endoluminal data points within the stack correspond to relative locations of the endoluminal data points with respect to the roadmap image.

29. The method according to claim 28, further comprising based upon the co-registering of the endoluminal data points to respective locations within the roadmap image, determining a parameter of a portion of the lumen that corresponds to a portion of the stack of endoluminal data points, and generating an output in response thereto.

30. The method according to claim 16, wherein designating at least one of the first set of images as a roadmap image comprises using the at least one processor to designate the at least one of the first set of images as the roadmap image at least partially in response to an input from a user.