Endolumenal object sizing

• US 11,337,602 B2
• Filed: 11/02/2018
• Issued: 05/24/2022
• Est. Priority Date: 12/28/2016
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Mintz, David S., Schummers, David M., Jeevan, Prasanth, Rafii-Tari, Hedyeh, Ummalaneni, Ritwik
• Primary Examiner(s): Luong, Peter
• Application Number:
  US16/179,568
• Publication Number:
  US 20190175009A1
• Time in Patent Office: 1,299 Days
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 1/06 with illuminating arrangements
• CPC Class Codes: A61B 1/307 for the urinary organs, e.g. urethroscopes, cystoscopes
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/303 specifically adapted for manipulations within body lumens, e.g. within lumen of gut, spine, or blood vessels
• CPC Class Codes: A61B 2090/061 for measuring dimensions, e.g. length
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/107 Measuring physical dimensions, e.g. size of the entire body or parts thereof
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation

Abstract

An object sizing system sizes an object positioned within a patient. The object sizing system identifies a presence of the object. The object sizing system navigates an elongate body of an instrument to a position proximal to the object within the patient. An imaging sensor coupled to the elongate body captures one or more sequential images of the object. The instrument may be further moved around within the patient to capture additional images at different positions/orientations relative to the object. The object sizing system also acquires robot data and/or EM data associated with the positions and orientations of the elongate body. The object sizing system analyzes the captured images based on the acquired robot data to estimate a size of the object.

Claims

1. A method for sizing an object within a patient, the method comprising:

navigating an elongate body to a location within the patient and proximate to the object, the elongate body comprising an imaging sensor for capturing images of the object and a basket for capturing the object;

capturing sequential images of the object and the basket with the imaging sensor while navigating the elongate body into at least one of a different orientation and a different position;

accessing, with a processor, a set of robot data indicative of the orientation and position of the elongate body during the capture of the sequential images, wherein the accessed set of robot data comprises data from which a position and size of the basket can be determined; and

analyzing, with the processor, the sequential images to estimate a size of the object based on;

identifying those images depicting at least a portion of the object and at least a portion of the basket; and

determining the estimated size of the object based on the position and the size of the basket relative to the object in the identified images.

2. The method of claim 1, wherein analyzing the sequential images comprises segmenting the sequential images to remove the basket from the analyzed images.

3. The method of claim 2, wherein analyzing the sequential images comprises determining a structure of the object based on the segmented images.

4. The method of claim 1, further comprising capturing the object with the basket.

5. The method of claim 4, wherein capturing the sequential images occurs before capturing the object with the basket.

6. The method of claim 4, wherein capturing the sequential images occurs after capturing the object with the basket.

7. The method of claim 4, wherein analyzing the sequential images comprises:

detecting a basket marker in the sequential images; and

determining a distance to the object based on the detected basket marker,

wherein determining the estimated size of the object is further based on the distance to the object.

8. The method of claim 1, wherein the set of robot data comprises at least one of data collected from a robotic manipulator and data collected from an electromagnetic sensor coupled to the elongate body.

9. The method of claim 1, further comprising inserting a guide wire into the patient to measure a distance from the imaging sensor to the object.

10. The method of claim 1, wherein analyzing the sequential images comprises:

combining the sequential images via image stitching techniques;

identifying a structure of the object;

determining an estimated size of the identified structure of the object; and

determining the estimated size of the object further based on the estimated size of the structure of the object.

11. The method of claim 10, wherein the structure of the object comprises at least one of an outer axial diameter and a radius of the object.

12. A system for sizing an object within a patient, the system comprising:

an elongate body comprising an imaging sensor for capturing images of the object and a basket for capturing the object; and

a computer comprising instructions that, when executed by a processor of the computer, cause the computer to;

navigate the elongate body into the patient and to a location proximate to the object, the navigating enabled by a robotic manipulator;

capture sequential images of the object and the basket with the imaging sensor while navigating the elongate body into at least one of a different orientation and a different position;

access a set of robotic data describing the changing orientation and position of the elongate body during the capture of the sequential images; and

analyze the sequential images to estimate a size of the object based on;

identifying those images depicting at least a portion of the object and at least a portion of the basket; and

determining the estimated size of the object based on the position and the size of the basket relative to the object in the identified images.

13. The system of claim 12, wherein the instructions, when executed by the processor, cause the computer to analyze the sequential images based on segmenting the sequential images to remove the basket from the analyzed images.

14. The system of claim 12, wherein the instructions, when executed by the processor, cause the computer to instruct the system to capture the object with the basket.

15. The system of claim 14, wherein the capturing of the sequential images occurs before the capturing of the object with the basket.

16. The system of claim 14, wherein the capturing of the sequential images occurs after the capturing of the object with the basket.

17. The system of claim 14, wherein instructions, when executed by theprocessor, cause the computer to analyze the sequential images based on:

detecting a basket marker in the sequential images; and

determining a distance to the object based on the detected basket marker, wherein determining the estimated size of the object is further based on the distance to the object.

18. The system of claim 12, wherein the set of robot data comprises at least one of data collected from a robotic manipulator and data collected from an electromagnetic sensor coupled to the elongate body.

19. The system of claim 12, further comprising:

a guide wire, wherein the instructions, when executed by the processor, cause the computer to;

insert the guide wire into the patient to measure a distance from the imaging sensor to the object.

20. The system of claim 12, wherein the instructions, when executed by theprocessor, cause the computer to analyze the sequential images based on:

combining the sequential images via image stitching techniques;

identifying a structure of the object; and

determining the estimated size of the object further based on the identified structure.