Quantitative three-dimensional imaging of surgical scenes from multiport perspectives

• US 10,334,227 B2
• Filed: 03/28/2015
• Issued: 06/25/2019
• Est. Priority Date: 03/28/2014
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Panescu, Dorin, Jones, Daniel H.
• Primary Examiner(s): Jiang, Zaihan
• Application Number:
  US15/300,265
• Publication Number:
  US 20170180704A1
• Time in Patent Office: 1,550 Days
• Field of Search: 348 45
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 17/00 Surgical instruments, devices or methods, e.g. tourniquets A61B18/00 takes precedence; contraceptive devices, pessaries, or applicators therefor A61F6/00; eye surgery A61F9/007; ear surgery A61F11/00
• CPC Class Codes: A61B 2017/00119 alarm; indicating an abnormal situation
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes
• CPC Class Codes: H04N 23/56 provided with illuminating means

Abstract

A system is provided that includes a quantitative three-dimensional (Q3D) endoscope that is inserted through a first port providing access to a body cavity to a first position at which a first 3D image of a first portion of an anatomical structure is captured from a perspective of said endoscope. A first Q3D model is produced based upon the captured first image. The endoscope is inserted through a second port providing access to the body cavity to a second position at which a second image of a second portion of the anatomical structure is captured. A second quantitative Q3D model is produced based upon the captured second image. The first Q3D model and the second Q3D model are combined together to produce an expanded Q3D model of the anatomical structure. The expanded Q3D model can be stored for further manipulation or can be displayed in 3D.

Claims

1. A system for 3D imaging of a surgical scene comprising:

a quantitative three-dimensional (Q3D) endoscope operable to image an anatomical structure within a body cavity;

at least one manipulator configured to insert the Q3D endoscope through a first port and later through at least a second port, the first port and the second port each providing access to the body cavity; and

at least one processor configured to;

produce a first Q3D model based upon a first image captured with the endoscope in the first port;

produce a second Q3D model based upon a second image captured with the endoscope in the second port; and

combine together the first Q3D model and the second Q3D model;

wherein the combining together includes;

determining a first position of the endoscope relative to a first coordinate system when the endoscope is in the first port;

determining a second position of the endoscope relative to a second coordinate system when the endoscope is in the second port;

using a kinematic relationship between the first position of the endoscope and the second position of the endoscope to align the first Q3D model and the second Q3D model.

3. The system of claim 1 further including:

a 3D display configured to display an expanded Q3D model.

4. The system of claim 1 further including:

means to manipulate an expanded Q3D model.

5. The system of claim 4 wherein the means to manipulate the expanded Q3D model are configured to execute quantitative measurements of distances between first and second surgical instruments or between a surgical instrument and an anatomic structure.

6. The system of claim 1 further including;

means to control the at least one manipulator to jitter a position of the endoscope to successive positions.

2. A system for 3D imaging of a surgical scene comprising:

a quantitative three-dimensional (Q3D) endoscope operable to image an anatomical structure within a body cavity;

at least one manipulator configured to insert the Q3D endoscope through a first port and later through at least a second port, the first port and the second port each providing access to the body cavity; and

at least one processor configured to;

produce a first Q3D model based upon a first image captured with the endoscope in the first port, produce a second Q3D model based upon a second image captured with the endoscope in the second port, and

combine together the first Q3D model and the second Q3D model;

wherein the combining together includes;

identifying common fiducial points within the first and second Q3D models; and

geometrically transforming relative positions of the first Q3D model and the second Q3D model to each other so as to align the identified common fiducial points.

16. The system of claim 2 further including:

a 3D display configured to display an expanded Q3D model.

17. The system of claim 2 further including:

means to manipulate an expanded Q3D model.

18. The system of claim 17 wherein the means to manipulate the Q3D model are configured to execute quantitative measurements of distances between first and second surgical instruments or between a surgical instrument and an anatomic structure.

19. The system of claim 2 further including:

means to control the at least one manipulator to jitter a position of the endoscope to successive positions.

7. A method for three-dimensional (3D) imaging of a surgical scene comprising:

illuminating an anatomical structure within a human body cavity with a light source;

inserting a quantitative three-dimensional (Q3D) endoscope that is coupled to a manipulator, through a first port providing access to a body cavity to image a first portion of the anatomical structure;

capturing a first image;

producing a first Q3D model based upon the first image,

inserting said endoscope that is coupled to a manipulator, through a second port providing access to same body cavity to image a second portion of the anatomical structure;

capturing a second image;

producing a second Q3D model based upon the second image; and

combining together the first Q3D model and the second Q3D model;

wherein combining includes;

determining a first position of the endoscope relative to a first coordinate system when inserted in the first port;

determining a second position of the endoscope relative to a second coordinate system when inserted in the second port; and

using a kinematic relationship between the first position of the endoscope and the second position of the endoscope to align the first Q3D model and the second Q3D model.

9. The method of claim 7 further including:

inserting said endoscope that is coupled to a manipulator, in a third port providing access to same body cavity to image a third portion of the anatomical structure;

capturing a third image;

producing a third quantitative three-dimensional (Q3D) model based upon the third image; and

combining together the third Q3D model with the first and second Q3D models to produce an expanded Q3D model.

10. The method of claim 7 wherein:

combining the first and second Q3D models consists of stitching the first and second Q3D models in order to produce an expanded Q3D model.

11. The method of claim 7 further including;

storing an expanded Q3D model for further manipulation by a user.

12. The method of claim 11,

wherein the further manipulation includes quantitative measurements of distances between other instruments or between instruments and anatomic structures.

13. The method of claim 7 further including:

displaying in 3D an expanded Q3D model.

14. The method of claim 7,

wherein the endoscope is first coupled to one of first and second manipulators and is later coupled to another of the first and second manipulators.

15. The method of claim 7,

wherein the endoscope is first inserted through one of the first port and the second port coupled to one of first and second manipulators and is later inserted through the other of the first port and the second port coupled to another of the first and second manipulators.

8. A method for three-dimensional (3D) imaging of a surgical scene comprising:

illuminating an anatomical structure within a human body cavity with a light source;

inserting a quantitative three-dimensional (Q3D) endoscope that is coupled to a manipulator, through a first port providing access to a body cavity to image a first portion of the anatomical structure;

capturing a first image;

producing a first Q3D model based upon the first image;

inserting said endoscope that is coupled to a manipulator, through a second port providing access to same body cavity to image a second portion of the anatomical structure;

capturing a second image;

producing a second Q3D model based upon the second image; and

combining together the first Q3D model and the second 03D model;

wherein combining includes;

identifying common fiducial points within the first and second Q3D models;

and geometrically transforming relative positions of the first Q3D model and the second Q3D model to each other so as to align the identified fiducial points.

20. The method of claim 8 further including:

inserting said endoscope that is coupled to a manipulator, in a third port providing access to same body cavity to image a third portion of the anatomical structure;

capturing a third image;

producing a third quantitative three-dimensional (Q3D) model based upon the third image; and

combining together the third Q3D model with the first and second Q3D models to produce an expanded Q3D model.

21. The method of claim 8 wherein:

combining the first and second Q3D models consists of stitching the first and second Q3D models in order to produce an expanded Q3D model.

22. The method of claim 8 further including:

storing the expanded Q3D model for further manipulation by a user.

23. The method of claim 22,

wherein the further manipulation includes quantitative measurements of distances between other installments or between instruments and anatomic structures.

24. The method of claim 8 further including:

displaying in 3D an expanded Q3D model.

25. The method of claim 8,

wherein the endoscope is first coupled to one of a first and second manipulators and is later coupled to another of the first and second manipulators.

26. The method of claim 8,

wherein the endoscope is first inserted through one of first port and the second port coupled to one of first and second manipulators and is later inserted through the other of the first port and the second port coupled to another of the first and second manipulators.