Quantitative three-dimensional imaging and printing of surgical implants

• US 10,350,009 B2
• Filed: 03/28/2015
• Issued: 07/16/2019
• Est. Priority Date: 03/28/2014
• Status: Active Grant

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Inventors: Panescu, Dorin, Jones, Daniel H
• Primary Examiner(s): Adams, Eileen M
• Application Number:
  US15/300,262
• Publication Number:
  US 20170181798A1
• Time in Patent Office: 1,571 Days
• Field of Search: None
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0004 for electronic operation
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: A61B 1/00055 for alerting the user
• CPC Class Codes: A61B 1/00089 Hoods
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/00165 with light-conductive means, e.g. fibre optics A61B1/07 takes precedence
• CPC Class Codes: A61B 1/00194 adapted for three-dimensional imaging
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/0655 Control therefor
• CPC Class Codes: A61B 1/0676 at distal tip of an endoscope
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 34/37 : Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 5/4851 : Prosthesis assessment or monitoring
• CPC Class Codes: G02B 23/2415 : Stereoscopic endoscopes
• CPC Class Codes: G02B 23/2484 : Arrangements in relation to a camera or imaging device processing or control of video signals generated by an endoscope H04N23/60, H04N23/70
• CPC Class Codes: G06T 17/00 : Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: H04N 13/232 : using fly-eye lenses, e.g. arrangements of circular lenses
• CPC Class Codes: H04N 13/254 : in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/30 : Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00

Abstract

A system to produce a replacement anatomical structure, comprising: a quantitative three-dimensional (Q3D) endoscope disposed to image a structure within a field of view that includes target tissue; at least one processor configured to: produce a first Q3D model of an anatomical structure that includes target tissue; produce a second Q3D model of the anatomical structure that includes remainder tissue in a location from which the target has been tissue removed; and produce a third Q3D model of a replacement structure based at least in part upon the first Q3D model and the second Q3D model.

Claims

1. A system to produce a replacement anatomical structure, comprising:

a quantitative three-dimensional (Q3D) endoscope including an imaging sensor array comprising a plurality of imaging sensors, wherein each imaging sensor includes a corresponding lens stack and a two-dimensional arrangement of pixels arranged in a focal plane of the lens stack, wherein at least three of the imaging sensors of the imaging sensor array are disposed to have coplanar fields of view that overlap, and wherein the corresponding lens stack of each imaging sensor creates a separate optical channel that resolves an image of an anatomical structure within a field of view that includes target tissue in a focal plane of the corresponding lens stack;

at least one processor configured to;

produce a first Q3D model based upon image information captured from the anatomical structure that represents an external surface of bone tissue including the target bone tissue;

produce a second Q3D model based upon image information captured from the anatomical structure that represents remainder tissue structure, which defines an empty region in a location of the anatomical structure from which the target tissue has been tissue removed; and

produce a third Q3D model of a replacement structure that is shaped and dimensioned to fit against the remainder structure that defines the empty region and to substantially fill the empty region based at least in part upon the first Q3D model and the second Q3D model.

2. The system of claim 1,

wherein producing the third Q3D model includes determining one or more differences between the first Q3D model and the second Q3D model.

3. The system of claim 1,

wherein producing the third Q3D model includes determining 3D digital subtraction between the first Q3D model and the second Q3D model.

4. The system of claim 1,

wherein the at least one processor is configured to:

convert the third Q3D model to a three dimensional (3D) model of the replacement structure in a format that is suitable for use by a 3D printing machine.

5. The system of claim 1,

wherein the at least one processor is configured to:

provide an output signal indicative of the size required for said replacement structure.

6. A method of producing a surgical replacement structure comprising:

using a Q3D endoscope to capture image information from an external surface of bone tissue that includes target bone tissue;

producing a first quantitative three dimensional (Q3D) model based upon the image information captured from the external surface of the bone tissue that includes the target bone tissue;

using the Q3D endoscope to capture image information from remainder tissue bone structure, which defines an empty region in a location of the bone tissue from which the target bone tissue has been tissue removed;

producing a second Q3D model based upon image information captured from the anatomical structure that represents the remainder tissue bone structure, which defines the empty region in the location of the bone tissue from which the target bone tissue has been tissue removed; and

producing a third Q3D model of a replacement structure shaped and dimensioned to fit against the remainder structure that defines the empty region and to substantially fill the empty region, produced based upon the first Q3D model and the second Q3D model.

7. The method of claim 6 further including:

wherein producing the third Q3D model includes determining a 3D digital subtraction between the first Q3D model and the second Q3D model.

8. The method of claim 6 further including:

converting the third Q3D model to a three dimensional (3D) model of the replacement structure in a format that is suitable for use by a 3D printing machine.

9. The method of claim 8 further including using a 3D printing machine configured using the 3D model to produce the replacement structure.

10. The method of claim 6 further including:

sizing an off-the-shelf replacement structure for use as an implant for replacing said removed tissue based at least in part upon the third Q3D model of the replacement structure.

11. The method of claim 6,

wherein producing the third Q3D model includes determining one or more differences between the first Q3D model and the second Q3D model.

12. A system for 3D imaging of surgical scenes comprising:

a Q3D endoscope;

a manipulator to move the Q3D endoscope by known amounts to successive redundant positions to provide separate viewpoints of a target tissue region;

means to capture redundant multiple successive Q3D images from said successive positions; and

a processor configured to produce a Q3D model of the surgical scene based upon the redundant multiple successive Q3D images.

13. A method comprising:

illuminating an anatomical structure with a light source;

successively moving a Q3D endoscope by known amounts to each of multiple successive redundant positions to provide separate viewpoints of a target tissue region within a body cavity;

capturing redundant multiple successive image projections from the target tissue using the Q3D endoscope at the multiple successive redundant positions; and

producing a Q3D model based upon the redundant multiple successive captured image projections.

14. The method of claim 13, wherein successively moving the endoscope includes successively changing physical position of the endoscope.

15. The method of claim 13, wherein successively moving the endoscope includes successively adjusting optical property of one or more lenses associated with the Q3D endoscope.

16. A system for quantitative evaluation of an implantable device comprising:

a quantitative three-dimensional (Q3D) endoscope including an imaging sensor array comprising a plurality of imaging sensors, wherein each imaging sensor includes a corresponding lens stack and a two-dimensional arrangement of pixels arranged in a focal plane of the lens stack, wherein at least three of the imaging sensors of the imaging sensor array are disposed to have coplanar fields of view that overlap, and wherein the corresponding lens stack of each imaging sensor creates a separate optical channel that resolves an image a structure within a field of view that is targeted to receive an implantable device in a focal plane of the corresponding lens stack;

at least one processor configured to;

produce a first Q3D model of said anatomical structure before implantation; and

produce an output signal indicative of the size required for implanted device;

produce a second Q3D model of the anatomical structure after implantation; and

produce at least one quantitative evaluation output based upon, at least in part, on the second Q3D model.

17. A system for quantitative evaluation of an implantable device comprising:

a quantitative three-dimensional (Q3D) endoscope including an imaging sensor array comprising a plurality of imaging sensors, wherein each imaging sensor includes a corresponding lens stack and a two-dimensional arrangement of pixels arranged in a focal plane of the lens stack, wherein at least three of the imaging sensors of the imaging sensor array are disposed to have coplanar fields of view that overlap; and

wherein the corresponding lens stack of each imaging sensor creates a separate optical channel that resolves an image, of an opening and closing of an anatomical valve within a field of view that is targeted to receive an implantable prosthetic valve device in a focal plane of the corresponding lens stack;

at least one processor configured to;

produce one or more a first Q3D models of opening and closing of said anatomical valve before implantation of the prosthetic valve device; and

produce an output signal indicative of

opening or closing size required of the prosthetic valve device.

18. A system for quantitative evaluation of an implantable device comprising:

a quantitative three-dimensional (Q3D) endoscope including an imaging sensor array comprising a plurality of imaging sensors, wherein each imaging sensor includes a corresponding lens stack and a two-dimensional arrangement of pixels arranged in a focal plane of the lens stack, wherein at least three of the imaging sensors of the imaging sensor array are disposed to have coplanar fields of view that overlap, and wherein the corresponding lens stack of each imaging sensor creates a separate optical channel that resolves an image, of an anatomical lumen within a field of view that is targeted to receive an implantable stent device in a focal plane of the corresponding lens stack;

at least one processor configured to;

produce one or more a first Q3D models indicating mean cross section area of said anatomical lumen before implantation of the stent device; and

produce an output signal indicative of mean stent cross-sectional required of the stent device to fit within the anatomical lumen.

19. A system for quantitative evaluation of an implantable device comprising:

a quantitative three-dimensional (Q3D) endoscope including an imaging sensor array comprising a plurality of imaging sensors, wherein each imaging sensor includes a corresponding lens stack and a two-dimensional arrangement of pixels arranged in a focal plane of the lens stack, wherein at least three of the imaging sensors of the imaging sensor array are disposed to have coplanar fields of view that overlap, and wherein the corresponding lens stack of each imaging sensor creates a separate optical channel that resolves an image, of a stent device within an anatomical lumen within a field of view in a focal plane of the corresponding lens stack;

at least one processor configured to;

produce one or more a first Q3D models indicating the stent device within an inner wall of the anatomical lumen anatomical lumen; and

produce an output signal indicative of fit of the stent to an inner wall of the anatomical lumen.