Endoscopic registration of vessel tree images

• US 10,453,174 B2
• Filed: 10/19/2012
• Issued: 10/22/2019
• Est. Priority Date: 10/26/2011
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Popovic, Aleksandra, Elhawary, Haytham, Hall, Christopher Stephen
• Primary Examiner(s): Sun, Jiangeng
• Application Number:
  US14/353,776
• Publication Number:
  US 20140301618A1
• Time in Patent Office: 2,559 Days
• Field of Search: None
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 7/0012 Biomedical image inspection

Abstract

An image registration system employs an endoscope and an endoscope controller. In operation, the endoscope generates an intra-operative endoscopic image of a vessel tree (e.g., an arterial tree or a venous tree) within an anatomical region, and the endoscope controller image registers the intra-operative endoscopic image of the vessel tree to a pre-operative three-dimensional image of the vessel tree within the anatomical region. The image registration includes an image matching of a graphical representation of each furcation of the vessel tree within the intra-operative endoscopic image of the vessel tree to a graphical representation of each furcation of the vessel tree within the pre-operative three-dimensional image of the vessel tree.

Claims

1. An image registration system, comprising:

an endoscope operable for generating an intra-operative endoscopic image of a vessel tree within an anatomical region; and

an endoscope controller structurally configured to image register the intra-operative endoscopic image of the vessel tree to a pre-operative three-dimensional image of the vessel tree responsive to a generation of the intra-operative endoscopic image of the vessel tree,

wherein an image registration by the endoscope controller includes an image matching by the endoscope controller of a graphical representation of each furcation of the vessel tree within the intra-operative endoscopic image of the vessel tree to a graphical representation of each furcation of the vessel tree within the pre-operative three-dimensional image of the vessel tree;

wherein the image matching by the endoscope controller includes;

a generation by the endoscope controller of a main graph derived from a geometrical representation of the pre-operative three-dimensional image of the vessel tree, the main graph including a main set of connected nodes representative of each furcation of the vessel tree within the pre-operative three-dimensional image of the vessel tree;

a generation by the endoscope controller of a subgraph derived from a geometrical representation of the intra-operative endoscopic image of the vessel tree, the subgraph including a subset of the main set of connected nodes and being representative of each furcation of the vessel tree within the intra-operative endoscopic image of the vessel tree; and

a node matching by the endoscope controller of the subgraph to the main graph, wherein the endoscope controller is further structurally configured to update the image registration of the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree responsive to any surgical alteration in a topology of the vessel tree;

wherein the main graph by the endoscope controller is modified to reflect a surgical alteration in the topology of the vessel tree; and

wherein the modifying of the arterial main graph by the endoscope controller includes at least one of (1) a connecting by the endoscope controller of an additional node to the main graph, the new node being representative of a surgical creation of an additional furcation of the vessel tree, and (2) a disconnecting by the endoscope controller of one of the nodes of the main graph, the disconnected node being representative of a surgical removal of one of the furcations of the vessel tree.

2. The image registration system of claim 1, wherein the endoscope controller is further structurally configured to update the image registration of the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree responsive to any surgical alteration in a topology of the vessel tree.

3. An image registration system, comprising:

an endoscope operable for generating an intra-operative endoscopic image of an anatomical region including an arterial tree and a venous tree; and

an endoscope controller structurally configured to image register the arterial tree within the intra-operative endoscopic image to a pre-operative three-dimensional image of the anatomical region responsive to a generation of the intra-operative endoscopic image of an anatomical region;

wherein the image registration by the endoscope controller includes a venous image matching by the endoscope controller of a graphical representation of each furcation of the venous tree within the intra-operative endoscopic image of the anatomical region to a graphical representation of each furcation of the venous tree within the pre-operative three-dimensional image of the anatomical region;

wherein the image registration by the endoscope controller includes an arterial image matching by the endoscope controller of a graphical representation of each furcation of the arterial tree within the intra-operative endoscopic image of the anatomical region to a graphical representation of each furcation of the arterial tree within the pre-operative three-dimensional image of the anatomical region;

wherein the venous image matching by the endoscope controller includes;

a generation by the endoscope controller of a venous main graph derived from a geometrical representation of the pre-operative three-dimensional image of the anatomical region, the venous main graph including a main set of venous nodes representative of each furcation of the venous tree within the pre-operative three-dimensional image of the anatomical region;

a generation by the endoscope controller of a venous subgraph derived from a geometrical representation of the intra-operative endoscopic image of the anatomical region, the venous subgraph including a subset of the main set of venous nodes and being representative of each furcation of the venous tree within the intra-operative endoscopic image of the anatomical region;

wherein the arterial image matching by the endoscope controller includes;

a generation by the endoscope controller of an arterial main graph derived from a geometrical representation of the pre-operative three-dimensional image of the anatomical region, the arterial main graph including a main set of arterial nodes representative of each furcation of the arterial tree within the pre-operative three-dimensional image of the anatomical region, and

a generation by the endoscope controller of an arterial subgraph derived from a geometrical representation of the intra-operative endoscopic image of the anatomical region, the arterial subgraph including a subset of the main set of arterial nodes and being representative of each furcation of the arterial tree within the intra-operative endoscopic image of the anatomical region;

wherein the endoscope controller is further structurally configured to update the image registration of the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree responsive to any surgical alteration in a topology of the vessel tree,

wherein the arterial main graph is modified by the endoscope controller to reflect a surgical alteration in the topology of the vessel tree; and

wherein the modifying of the arterial main graph by the endoscope controller includes a connecting by the endoscope controller of an additional node to the main graph, the new node being representative of a surgical creation of an additional furcation of the vessel tree.

4. The image registration system of claim 3, wherein the endoscope controller is further structurally configured to update the image registration of the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree responsive to any surgical alteration in a topology of the vessel tree.

5. The image registration system of claim 4, wherein the image registration by the endoscope controller further includes a determination by the endoscope controller of a relative positioning of the arterial tree in the pre-operative three-dimensional image of the anatomical region to the venous tree in the pre-operative three-dimensional image of the anatomical region.

6. The image registration system of claim 3, wherein the venous image matching by the endoscope controller includes:

a generation by the endoscope controller of a venous main graph derived from a geometrical representation of the pre-operative three-dimensional image of the anatomical region, the venous main graph including a main set of venous nodes representative of each furcation of the venous tree within the pre-operative three-dimensional image of the anatomical region, and

a generation by the endoscope controller of a venous subgraph derived from a geometrical representation of the intra-operative endoscopic image of the anatomical region, the venous subgraph including a subset of the main set of venous nodes and being representative of each furcation of the venous tree within the intra-operative endoscopic image of the anatomical region.

7. The image registration system of claim 3,

wherein the venous image matching by the endoscope controller further includes a venous node matching by the endoscope controller of the venous subgraph to the venous main graph;

wherein the arterial image matching by the endoscope controller further includes an arterial node matching by the endoscope controller of the arterial subgraph to the arterial main graph; and

wherein the image registration further includes a combination of the venous node matching and the arterial node matching.

8. The image registration system of claim 3,

wherein the image registration by the endoscope controller further includes an integration by the endoscope controller of the venous main graph and the arterial main graph;

wherein the venous image matching by the endoscope controller further includes a venous node matching by the endoscope controller of the venous subgraph to the integration of the venous main graph and the arterial main graph; and

wherein the arterial image matching by the endoscope controller further includes an arterial node matching by the endoscope controller of the arterial subgraph to the integration of the venous main graph and the arterial main graph.

9. An image registration system, comprising:

an endoscope operable for generating an intra-operative endoscopic image of an anatomical region including an arterial tree and a venous tree; and

an endoscope controller structurally configured to image register the arterial tree within the intra-operative endoscopic image to a pre-operative three-dimensional image of the anatomical region responsive to a generation of the intra-operative endoscopic image of an anatomical region,

wherein the image registration by the endoscope controller includes a venous image matching by the endoscope controller of a graphical representation of each furcation of the venous tree within the intra-operative endoscopic image of the anatomical region to a graphical representation of each furcation of the venous tree within the pre-operative three-dimensional image of the anatomical region;

wherein the image registration by the endoscope controller includes an arterial image matching by the endoscope controller of a graphical representation of each furcation of the arterial tree within the intra-operative endoscopic image of the anatomical region to a graphical representation of each furcation of the arterial tree within the pre-operative three-dimensional image of the anatomical region,

wherein the venous image matching by the endoscope controller includes;

a generation by the endoscope controller of a venous main graph derived from a geometrical representation of the pre-operative three-dimensional image of the anatomical region, the venous main graph including a main set of venous nodes representative of each furcation of the venous tree within the pre-operative three-dimensional image of the anatomical region, and

a generation by the endoscope controller of a venous subgraph derived from a geometrical representation of the intra-operative endoscopic image of the anatomical region, the venous subgraph including a subset of the main set of venous nodes and being representative of each furcation of the venous tree within the intra-operative endoscopic image of the anatomical region;

wherein the arterial image matching by the endoscope controller includes;

a generation by the endoscope controller of an arterial main graph derived from a geometrical representation of the pre-operative three-dimensional image of the anatomical region, the arterial main graph including a main set of arterial nodes representative of each furcation of the arterial tree within the pre-operative three-dimensional image of the anatomical region, and

a generation by the endoscope controller of an arterial subgraph derived from a geometrical representation of the intra-operative endoscopic image of the anatomical region, the arterial subgraph including a subset of the main set of arterial nodes and being representative of each furcation of the arterial tree within the intra-operative endoscopic image of the anatomical region;

wherein the endoscope controller is further structurally configured to update the image registration of the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree responsive to any surgical alteration in a topology of the vessel tree;

wherein the arterial main graph is modified by the endoscope controller to reflect a surgical alteration in the topology of the vessel tree; and

wherein the modifying of the arterial main graph by the endoscope controller includes a disconnecting by the endoscope controller of one of the nodes of the main graph, the disconnected node being representative of a surgical removal of one of the furcations of the vessel tree.

10. An image registration method, comprising:

generating a pre-operative three-dimensional image of a vessel tree within an anatomical region;

generating an intra-operative endoscopic image of the vessel tree within the anatomical region; and

an endoscope controller image registering the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree,

wherein the image registration by the endoscope controller includes an image matching by the endoscope controller of a graphical representation of each furcation of the vessel tree within the intra-operative endoscopic image of the vessel tree to a graphical representation of each furcation of the vessel tree within the pre-operative three-dimensional image of the vessel tree,

the endoscope controller updating the image registration of the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree responsive to a surgical alteration the vessel tree;

wherein a main graph is modified by the endoscope controller to reflect a change in the topology of the vessel tree;

wherein the modifying of the arterial main graph by the endoscope controller includes (1) a connecting by the endoscope controller of an additional node to the main graph, the new node being representative of a surgical creation of an additional furcation of the vessel tree or (2) a disconnecting by the endoscope controller of one of the nodes of the main graph, the disconnected node being representative of a surgical removal of one of the furcations of the vessel tree.

11. The image registration method of claim 10, further comprising

the endoscope controller updating the image registration of the intra-operative endoscopic image of the vessel tree to the pre-operative three-dimensional image of the vessel tree responsive to any surgical variation on the vessel tree.

12. The image registration method of claim 10, wherein the image matching by the endoscope controller includes:

the endoscope controller generating the main graph derived from a geometrical representation of the pre-operative three-dimensional image of the vessel tree, the main graph including a main set of connected nodes representative of each furcation of the vessel tree within the pre-operative three-dimensional image of the vessel tree; and

the endoscope controller generating a subgraph derived from a geometrical representation of the intra-operative endoscopic image of the vessel tree, the subgraph including a subset of the main set of connected nodes and being representative of each furcation of the vessel tree within the intra-operative endoscopic image of the vessel tree; and

the endoscope controller node matching the subgraph to the main graph.

13. The image registration method of claim 10, wherein the vessel tree is one of an arterial tree and a venous tree.