Robotic control of an endoscope from blood vessel tree images

• US 10,182,704 B2
• Filed: 04/10/2017
• Issued: 01/22/2019
• Est. Priority Date: 09/15/2010
• Status: Active Grant

• Current Assignee: Koninklijke Philips N.V.
• Original Assignee: Koninklijke Philips N.V.
• Inventors: Popovic, Aleksandra
• Primary Examiner(s): Leubecker, John P
• Assistant Examiner(s): CHOU, WILLIAM B
• Application Number:
  US15/483,615
• Publication Number:
  US 20170209028A1
• Time in Patent Office: 652 Days
• Field of Search: 600102, 600109, 600114, 600117, 600118, 600160
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/3137 for examination of the interior of blood vessels
• CPC Class Codes: A61B 2017/00252 for by-pass connections, i.e. connections from heart chamber to blood vessel or from blood vessel to blood vessel
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2090/3614 using optical fibre
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/10072 Tomographic images
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/30101 Blood vessel; Artery; Vein; Vascular
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/75 involving models

Abstract

A robot guiding system employs a robot unit (10) and a control unit (20). The robot unit (10) includes an endoscope (12) for generating an intra-operative endoscopic image (14) of a blood vessel tree within an anatomical region, and a robot (11) for moving the endoscope (12) within the anatomical region. The control unit (20) includes an endoscope controller (22) for generating an endoscopic path within the anatomical region, wherein the endoscopic path is derived from a matching of a graphical representation of the intra-operative endoscopic image (14) of the blood vessel tree to a graphical representation of a pre-operative three-dimensional image (44) of the blood vessel tree. The control unit (20) further includes a robot controller (21) for commanding the robot (11) to move the endoscope (12) within the anatomical region in accordance with the endoscopic path.

Claims

1. A control unit for controlling a surgical robot guiding system including an endoscope and a robot configured to move the endoscope within an anatomical region exterior to blood vessels of a blood vessel tree, the control unit comprising:

an endoscope controller configured to generate an endoscopic path exterior to the blood vessels within the anatomical region, wherein the endoscopic path is derived from matching a geometrical representation of an intra-operative endoscopic image of a subregion of the blood vessel tree to a geometrical representation of a pre-operative three-dimensional image of the blood vessel tree, wherein the matching of the geometrical representation of the intra-operative endoscopic image of the blood vessel tree to the geometrical representation of the pre-operative three-dimensional image of the blood vessel tree includes;

generating a main graph derived from the geometrical representation of the pre-operative three-dimensional image of the blood vessel tree, wherein the main graph includes a main set of nodes representative of each furcation of the blood vessel tree within the pre-operative three-dimensional image of the blood vessel tree and the nodes of the main graph are directionally linked with the varying diameters of the branches of the blood vessel tree indicating orientation of the directional links,

generating a subgraph derived from the geometrical representation of the intra-operative endoscopic image of the subregion of the blood vessel tree, wherein the subgraph includes a subset of the main set of nodes, the subset of nodes being representative of each furcation of the blood vessel tree within the intra-operative endoscopic image of the blood vessel tree, and

matching the subgraph to the main graph using the directional links of the main graph to reduce possible matches of nodes of the subgraph to the main graph in the matching; and

a robot controller configured to command the robot to move the endoscope within the anatomical region exterior to the blood vessels in accordance with the endoscopic path.

2. The control unit of claim 1, wherein the blood vessel tree is a coronary artery tree.

3. A control unit for controlling a surgical robot guiding system including an endoscope and a robot configured to move the endoscope within an anatomical region exterior to blood vessels of a blood vessel tree, the control unit comprising:

an endoscope controller configured to generate an endoscopic path exterior to the blood vessels within the anatomical region, wherein the endoscopic path is derived from matching a geometrical representation of an intra-operative endoscopic image of a subregion of the blood vessel tree to a geometrical representation of a pre-operative three-dimensional image of the blood vessel tree, wherein the matching of the geometrical representation of the intra-operative endoscopic image of the blood vessel tree to the geometrical representation of the pre-operative three-dimensional image of the blood vessel tree includes;

generating a main graph derived from the geometrical representation of the pre-operative three-dimensional image of the blood vessel tree, wherein the main graph includes a main set of nodes representative of each furcation of the blood vessel tree within the pre-operative three-dimensional image of the blood vessel tree,

generating a subgraph derived from the geometrical representation of the intra-operative endoscopic image of the subregion of the blood vessel tree, wherein the subgraph includes a subset of the main set of nodes, the subset of nodes being representative of each furcation of the blood vessel tree within the intra-operative endoscopic image of the blood vessel tree, and

matching of the subgraph to the main graph including establishing at least one of a vertical ordering and a horizontal ordering of the nodes in the main graph; and

a robot controller configured to command the robot to move the endoscope within the anatomical region exterior to the blood vessels in accordance with the endoscopic path.

4. The control unit of claim 3 wherein the matching of the subgraph to the main graph includes establishing a vertical ordering of the nodes in the main graph and using the vertical ordering to reduce possible matches of nodes of the subgraph to the main graph in the matching.

5. The control unit of claim 3 wherein the matching of the subgraph to the main graph includes establishing a horizontal ordering of the nodes in the main graph and using the horizontal ordering to reduce possible matches of nodes of the subgraph to the main graph in the matching.

6. A control unit for controlling a surgical robot guiding system including an endoscope and a robot configured to move the endoscope within an anatomical region exterior to blood vessels of a blood vessel tree, the control unit comprising:

an endoscope controller configured to generate an endoscopic path exterior to the blood vessels within the anatomical region, wherein the endoscopic path is derived from matching a geometrical representation of an intra-operative endoscopic image of a subregion of the blood vessel tree to a geometrical representation of a pre-operative three-dimensional image of the blood vessel tree; and

a robot controller configured to command the robot to move the endoscope within the anatomical region exterior to the blood vessels in accordance with the endoscopic path

wherein the matching of the geometrical representation of the intra-operative endoscopic image of the blood vessel tree to the geometrical representation of the pre-operative three-dimensional image of the blood vessel tree includes;

generating a main graph derived from the geometrical representation of the pre-operative three-dimensional image of the blood vessel tree;

generating a subgraph derived from the geometrical representation of the intra-operative endoscopic image of the subregion of the blood vessel tree; and

matching the subgraph to the main graph

wherein the endoscope controller is further configured to overlay the geometrical representation of the pre-operative three-dimensional image of the exterior of the blood vessel tree onto the intra-operative endoscopic image of the subregion of the blood vessel tree corresponding to the subgraph in accordance with the matching of the geometrical representation of the intra-operative endoscopic image of the blood vessel tree to the geometrical representation of the pre-operative three-dimensional image of the blood vessel tree.

7. The control unit of claim 6, wherein the endoscope controller is further configured to generate the endoscopic path within the geometrical representation of the pre-operative three-dimensional image of the exterior of the blood vessel tree overlaid onto the intra-operative endoscopic image of the corresponding portion of the blood vessel tree.

8. The control unit of claim 6, wherein the matching of the subgraph to the main graph includes matching nodes of the subgraph to nodes of subregions of the main graph to locate one or more candidate subregions of the main graph corresponding to the subgraph;

and

wherein one of the plurality of the candidate subregions located from the matching is selected as a match of the subgraph to the main graph.

9. The control unit of claim 6, wherein the blood vessel tree is a coronary artery tree.