Coregistering images of a region of interest during several conditions using a landmark subsurface feature

• US 8,896,678 B2
• Filed: 09/16/2011
• Issued: 11/25/2014
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: Modulated Imaging, Inc.
• Original Assignee: The Invention Science Fund I, L.L.C. (Intellectual Ventures LLC)
• Inventors: Hyde, Roderick A., Kare, Jordin T., Leuthardt, Eric C., Lieberman, Erez, Rivet, Dennis J., Sweeney, Elizabeth A., Wood, Lowell L. Jr.
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): ELAHI, SHAN E
• Application Number:
  US13/200,107
• Publication Number:
  US 20130070069A1
• Time in Patent Office: 1,166 Days
• Field of Search: 348/65
• US Class Current: 348/65
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 6/506 for diagnosis of nerves
• CPC Class Codes: A61B 6/5241 combining overlapping images of the same imaging modality, e.g. by stitching
• CPC Class Codes: A61B 8/5253 combining overlapping images, e.g. spatial compounding
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30028 Colon; Small intestine
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass
• CPC Class Codes: H04N 1/00 Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Described embodiments include a system, method, and computer program product. A described system includes an image coregistration circuit that coregisters a first depiction of a region of interest of a mammalian body part during a first condition by a reference medical image and a second depiction of the region of interest of the mammalian body part during a second condition by a target medical image. The coregistration is at least partially based on the first spatial relationship and on the second spatial relationship. The described system includes a computer-readable media configured to maintain informational data corresponding to the coregistration of the first depiction of the region of interest and the second depiction of the region of interest.

Claims

1. A computer program product comprising:

(a) a non-transitory computer readable medium including program instructions stored thereon which, when executed by a processor of a computing device, cause the computing device to perform a process including;

(i) coregistering

(x) a first depiction by a reference medical image of a region of interest of a mammalian body part during a first condition, the region of interest having a first spatial relationship to a landmark subsurface feature of the mammalian body part during the first condition, and

(y) a second depiction by a target medical image of the region of interest of the mammalian body part during a second condition, the region of interest having a second spatial relationship to the landmark subsurface feature of the mammalian body part during the second condition, the second spatial relationship being different than the first spatial relationship,

the coregistration of the first depiction and the second depiction at least partially based on the first spatial relationship and on the second spatial relationship;

(ii) storing in another non-transitory computer-readable media operably coupled with processor informational data corresponding to the coregistration of the first depiction of the region of interest and the second depiction of the region of interest.

2. A system comprising:

(a) a receiver circuit configured to receive;

(i) a reference medical image that includes a first depiction of a region of interest of a mammalian body part during a first condition;

(ii) a target medical image that includes a second depiction the region of interest of the mammalian body part during a second condition;

(iii) a first data indicative of a first spatial relationship between a landmark subsurface feature of the mammalian body part and the region of interest during the first condition; and

(iv) a second data indicative of a second spatial relationship between the landmark subsurface feature of the mammalian body part and the region of interest during the second condition, the second spatial relationship being different than the first spatial relationship;

(b) an image coregistration circuit configured to coregister the first depiction of the region of interest and the second depiction of the region of interest, the coregistration at least partially based on the first spatial relationship and on the second spatial relationship; and

(c) a non-transitory computer-readable medium configured to maintain informational data corresponding to the coregistration of the first depiction of the region of interest and the second depiction of the region of interest.

3. The system of claim 2, wherein the receiver circuit includes:

a receiver circuit configured to receive;

(i) a reference medical image that includes a first depiction of a region of interest of a cavity or lumen of a mammalian body part during a first condition;

(ii) a target medical image that includes a second depiction the region of interest of the cavity or lumen of the mammalian body part during a second condition;

(iii) a first data depicting a first spatial relationship between a landmark subsurface feature of the cavity or lumen of the mammalian body part and the region of interest during the first condition; and

(iv) a second data depicting a second spatial relationship between the landmark subsurface feature of the cavity or lumen of the mammalian body part and the region of interest during the second condition.

4. The system of claim 2, wherein the receiver circuit includes:

a receiver circuit configured to receive;

(i) a reference medical image that includes a first depiction of a region of interest of a surface of a cavity or lumen of a mammalian body part during a first condition;

(ii) a target medical image that includes a second depiction the region of interest of the surface of the cavity or lumen of the mammalian body part during a second condition;

(iii) a first data indicative of a first spatial relationship between a landmark subsurface feature of the surface of the cavity or lumen of the mammalian body part and the region of interest during the first condition; and

(iv) a second data indicative of a second spatial relationship between the landmark subsurface feature of the surface of the cavity or lumen of the mammalian body part and the region of interest during the second condition.

5. The system of claim 2, wherein the first receiver circuit includes:

a first receiver circuit configured to receive;

(i) a reference medical image acquired by a first device and including a first depiction of a region of interest of a mammalian body part during a first condition;

(ii) a target medical image acquired by a second device and including a second depiction the region of interest of the mammalian body part during a second condition;

(iii) a first data acquired by a third device and indicative of a first spatial relationship between a landmark subsurface feature of the mammalian body part and the region of interest during the first condition; and

(iv) a second data acquired by a fourth device and indicative of a second spatial relationship between the landmark subsurface feature of the mammalian body part and the region of interest during the second condition.

6. The system of claim 5, wherein the first device includes a body-insertable device.

7. The system of claim 5, wherein the first device includes an ex vivo device.

8. The system of claim 5, wherein the second device includes a device that is the same type as the first device.

9. The system of claim 5, wherein the second device includes a device of a different type than the first device.

10. The system of claim 5, wherein the third device includes a device that is the same type as the first device.

11. The system of claim 5, wherein the third device includes a device of a different type from the first device.

12. The system of claim 5, wherein the fourth device includes a device that is the same type as the first device.

13. The system of claim 5, wherein the fourth device includes a device of a different type from the first device.

14. The system of claim 2, wherein the image coregistration circuit includes:

an image coregistration circuit configured to coregister the location of the first depiction of the region of interest relative to the second depiction of the region of interest, the coregistration at least partially based on the first spatial relationship and the second spatial relationship.

15. The system of claim 2, wherein the image coregistration circuit includes:

an image coregistration circuit configured to coregister the first depiction of the region of interest and the second depiction of the region of interest, the coregistration including relative orientations of the first depiction of the region of interest and the second depiction of the region of interest, the coregistration at least partially based on the first spatial relationship and the second spatial relationship.

16. The system of claim 2, wherein the image coregistration circuit includes:

an image coregistration circuit configured to coregister the first depiction of the region of interest and the second depiction of the region of interest, the coregistration including a registration of relative distortions of the region of interest between the first depiction of the region of interest and the second depiction of the region of interest.

17. The system of claim 2, wherein the image coregistration circuit includes:

an image coregistration circuit configured to coregister the first depiction of the region of interest and the second depiction of the region of interest, the coregistration including a spatial translation of the second depiction of the region of interest to substantially align with the first depiction of the region of interest, the coregistration at least partially based on the first spatial relationship and the second spatial relationship.

18. The system of claim 2, wherein the image coregistration circuit includes:

an image coregistration circuit configured to coregister the first depiction of the region of interest and the second depiction of the region of interest, the coregistration at least partially based on a registration of the first depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part and on a registration of the second depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part.

19. The system of claim 2, further comprising:

a coordinate analysis circuit configured to determine a spatial relationship of the first depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part, and to determine a spatial relationship of the second depiction of the region of interest to the landmark subsurface feature of the mammalian body part.

20. The system of claim 19, wherein the image coregistration circuit includes:

an image coregistration circuit configured to coregister the first depiction of the region of interest and the second depiction of the region of interest, the coregistration at least partially based on the determined spatial relationship of the first depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part, and the determined spatial relationship of the second depiction of the region of interest to the landmark subsurface feature of the mammalian body part.

21. The system of claim 2, wherein the non-transitory computer-readable media includes:

a non-transitory computer-readable media configured to maintain and provide access to informational data corresponding to the coregistration of the first depiction of the region of interest and the second depiction of the region of interest.

22. The system of claim 2, further comprising:

a communication circuit configured to output a signal usable in displaying a human-perceivable indication of the informational data.

23. A method implemented in a computing device, the method comprising:

(a) receiving a reference medical image that includes a first depiction of a region of interest of a mammalian body part during a first condition;

(b) receiving a target medical image that includes a second depiction the region of interest of the mammalian body part during a second condition;

(c) receiving a first data indicative of a first spatial relationship between a landmark subsurface feature of the mammalian body part and the region of interest during the first condition;

(d) receiving a second data indicative of a second spatial relationship between the landmark subsurface feature of the mammalian body part and the region of interest during the second condition, the second spatial relationship being different than the first spatial relationship;

(e) coregistering the first depiction of the region of interest and the second depiction of the region of interest, the coregistration at least partially based on the first spatial relationship and the second spatial relationship; and

(f) maintaining in a non-transitory computer-readable media informational data corresponding to the coregistration of the first depiction of the region of interest and the second depiction of the region of interest.

24. The method of claim 23, wherein the mammalian body part includes a cavity or lumen of a mammalian body part.

25. The method of claim 23, wherein the mammalian body part includes a surface of a cavity or lumen of a mammalian body part.

26. The method of claim 23, wherein the receiving a reference medical image includes:

receiving a reference medical image acquired by a body-insertable device and including a first depiction of a region of interest of a mammalian body part during a first condition.

27. The method of claim 23, wherein the receiving a reference medical image includes:

receiving a reference medical image acquired by an ex vivo device and including a first depiction of a region of interest of a mammalian body part during a first condition.

28. The method of claim 23, wherein the receiving a target medical image includes:

receiving a target medical image acquired by a body-insertable device and including a second depiction the region of interest of the mammalian body part during a second condition.

29. The method of claim 23, wherein the receiving a target medical image includes:

receiving a target medical image acquired by an ex vivo device and including a second depiction the region of interest of the mammalian body part during a second condition.

30. The method of claim 23, wherein the receiving a first data includes:

receiving a first data acquired by a body-insertable device and indicative of a first spatial relationship between a landmark subsurface feature of the mammalian body part and the region of interest during the first condition.

31. The method of claim 23, wherein the receiving a first data includes:

receiving a first data acquired by an ex vivo device and indicative of a first spatial relationship between a landmark subsurface feature of the mammalian body part and the region of interest during the first condition.

32. The method of claim 23, wherein the receiving a second data includes:

receiving a second data acquired by a body-insertable device and indicative of a second spatial relationship between the landmark subsurface feature of the mammalian body part and the region of interest during the second condition.

33. The method of claim 23, wherein the receiving a second data includes:

receiving a second data acquired by an ex vivo device and indicative of a second spatial relationship between the landmark subsurface feature of the mammalian body part and the region of interest during the second condition.

34. The method of claim 23, wherein the coregistering includes:

spatially transforming the region of interest depicted by the target digital image to substantially align with the region of interest depicted by the reference image, the spatially transforming at least partially based on the first spatial relationship and the second spatial relationship.

35. The method of claim 23, wherein the coregistering includes:

coregistering the first depiction of the region of interest and the second depiction of the region of interest, the coregistration at least partially based on the first spatial relationship or the second spatial relationship.

36. The method of claim 23, further including:

registering both the first depiction of the region of interest and the second depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part, the registering at least partially based on the first spatial relationship and the second spatial relationship.

37. The method of claim 36, wherein the coregistering the first depiction of the region of interest and the second depiction of the region of interest includes:

coregistering the first depiction of the region of interest and the second depiction of the region of interest, the coregistering at least partially based upon the registering both the first depiction of the region of interest and the second depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part.

38. The method of claim 23, further including:

determining the first spatial relationship of the first depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part, and determining the second spatial relationship of the second depiction of the region of interest to the landmark subsurface feature of the mammalian body part.

39. The method of claim 38, the coregistering the first depiction of the region of interest and the second depiction of the region of interest includes:

registering both the first depiction of the region of interest and the second depiction of the region of interest relative to the landmark subsurface feature of the mammalian body part, the registering at least partially based on the determined first spatial relationship and the determined second spatial relationship.

40. The method of claim 23, wherein the maintaining in a non-transitory computer-readable media includes:

maintaining in a non-transitory computer-readable media and providing electronic access to informational data corresponding to the coregistration of the first depiction of the region of interest and the second depiction of the region of interest.

41. The method of claim 23, further comprising:

outputting a signal usable in displaying a human-perceivable indication of the informational data.

42. The method of claim 23, further comprising:

transforming the informational data into a particular visual representation of the coregistration of the first depiction of the region of interest and the second depiction of the region of interest.

43. The method of claim 23, further comprising:

transforming the informational data into a particular visual representation of the relative spatial relationships of the first depiction of the region of interest and the second depiction of the region of interest.

44. The method of claim 23, further comprising:

outputting the informational data.

45. The method of claim 23, further comprising:

providing a notification at least partially based on the informational data to at least one of a human, computer, or system.