SYSTEMS AND METHODS FOR DISPLAYING ESTIMATED LOCATION OF INSTRUMENT

• US 20190298160A1
• Filed: 03/26/2019
• Published: 10/03/2019
• Est. Priority Date: 03/28/2018
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Auris Health, Inc. (Johnson & Johnson)
• Inventors: Ummalaneni, Ritwik, Jeevan, Prasanth
• Granted Patent: US 10,827,913 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/0016 using motor drive units
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/303 specifically adapted for manipulations within body lumens, e.g. within lumen of gut, spine, or blood vessels
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2217/005 with suction drainage system
• CPC Class Codes: A61B 2217/007 with irrigation system
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 5/066 : Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 90/30 : Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: A61G 13/04 : tiltable around transverse or longitudinal axis
• CPC Class Codes: A61G 13/06 : raising or lowering of the whole table surface A61G13/04 takes precedence
• CPC Class Codes: A61G 13/08 : the table being divided into different adjustable sections
• CPC Class Codes: A61G 13/101 : Clamping means for connecting accessories to the operating table supports as part of surgical instruments A61B90/50

Abstract

Provided are systems and methods for displaying an estimated location of an instrument. In one aspect, the method includes determining a first location of the instrument based on first location data generated by a set of one or more location sensors for the instrument, the first location data corresponding to a first time period, and after the first time period, receiving a user command to move the instrument during a second time period. The method also includes estimating a second location of the instrument based on the first location and the received user command, the estimated second location corresponding to the second time period, and confirming the estimated second location based on second location data generated by the set of location sensors. The method further includes causing the estimated second location to be displayed prior to the confirmation of the estimated second location.

Claims

1. A system, comprising:

a processor; and

at least one computer-readable memory in communication with the processor and having stored thereon computer-executable instructions to cause the processor to;

determine a first location of an instrument based on first location data generated by a set of one or more location sensors for the instrument, the first location data corresponding to a first time period;

after the first time period, receive a user command to move the instrument during a second time period;

estimate a second location of the instrument based on the first location and the received user command, the estimated second location corresponding to the second time period;

confirm the estimated second location based on second location data generated by the set of location sensors, the second location data corresponding to the second time period; and

cause the estimated second location to be displayed prior to the confirmation of the estimated second location.

2. The system of claim 1, wherein the difference between the first time period and the second time period is based on a processing period for the confirmation of the second location estimate.

3. The system of claim 1, wherein:

the estimating of the second location has a first processing delay;

the confirming of the estimated second location has a second processing delay; and

the first processing delay is less than the second processing delay.

4. The system of claim 1, wherein the memory further has stored thereon computer-executable instructions to cause the processor to:

estimate the first location of the instrument for the first time period; and

confirm the estimated first location based on the first location data to determine the first location.

5. The system of claim 1, wherein the memory further has stored thereon computer-executable instructions to cause the processor to:

maintain a set of predictive states for a predicted location of the instrument, each of the predictive states associated with a corresponding probability; and

estimate the second location based on the predictive state having the highest probability for the second time period.

6. The system of claim 5, wherein the memory further has stored thereon computer-executable instructions to cause the processor to:

adjust a speed of movement of the instrument in response to the probability of the predictive state being less than a threshold probability.

7. The system of claim 1, wherein the memory further has stored thereon computer-executable instructions to cause the processor to:

receive second location data from set of the location sensors during the second time period, wherein the confirming of the estimated second location comprises determining the second location of the instrument based on the second location data; and

cause the estimated second location to be displayed prior to determining the second location based on the second location data.

8. The system of claim 7, wherein the memory further has stored thereon computer-executable instructions to cause the processor to:

compare the second location to the estimated second location; and

store the second location in the memory in response to the second location being different from the estimated second location.

9. The system of claim 8, wherein the memory further has stored thereon computer-executable instructions to cause the processor to:

estimate a third location of the instrument based on the estimated second location, the estimated third location corresponding to a third time period after the second time period; and

update the estimated third location based on the second location in response to the second location being different from the estimated second location.

10. The system of claim 1, wherein the set of location sensors comprises at least one of an electromagnetic (EM) sensor, a shape-sensing fiber, and a camera.

11. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:

determine a first location of an instrument based on first location data generated by a set of one or more location sensors for the instrument, the first location data corresponding to a first time period;

after the first time period, receive a user command to move the instrument during a second time period;

estimate a second location of the instrument based on the first location and the received user command, the estimated second location corresponding to the second time period;

confirm the estimated second location based on second location data generated by the set of location sensors, the second location data corresponding to the second time period; and

cause the estimated second location to be displayed prior to the confirmation of the estimated second location.

12. The non-transitory computer readable storage medium of claim 11, wherein the difference between the first time period and the second time period is based on a processing period for the confirmation of the second location estimate.

13. The non-transitory computer readable storage medium of claim 11, wherein:

the estimating of the second location has a first processing delay;

the confirming of the estimated second location has a second processing delay; and

the first processing delay is less than the second processing delay.

14. The non-transitory computer readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause at least one computing device to:

estimate the first location of the instrument for the first time period; and

confirm the estimated first location based on the first location data to determine the first location.

15. The non-transitory computer readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause at least one computing device to:

maintain a set of predictive states for a predicted location of the instrument, each of the predictive states associated with a corresponding probability; and

estimate the second location based on the predictive state having the highest probability for the second time period.

16. The non-transitory computer readable storage medium of claim 15, further having stored thereon instructions that, when executed, cause at least one computing device to:

adjust a speed of movement of the instrument in response to the probability of the predictive state being less than a threshold probability.

17. The non-transitory computer readable storage medium of claim 11, further having stored thereon instructions that, when executed, cause at least one computing device to:

receive second location data from set of the location sensors during the second time period, wherein the confirming of the estimated second location comprises determining the second location of the instrument based on the second location data; and

cause the estimated second location to be displayed prior to determining the second location based on the second location data.

18. The non-transitory computer readable storage medium of claim 17, further having stored thereon instructions that, when executed, cause at least one computing device to:

compare the second location to the estimated second location; and

store the second location in the memory in response to the second location being different from the estimated second location.

19. The non-transitory computer readable storage medium of claim 18, further having stored thereon instructions that, when executed, cause at least one computing device to:

estimate a third location of the instrument based on the estimated second location, the estimated third location corresponding to a third time period after the second time period; and

update the estimated third location based on the second location in response to the second location being different from the estimated second location.

20. The non-transitory computer readable storage medium of claim 11, wherein the set of location sensors comprises at least one of an electromagnetic (EM) sensor, a shape-sensing fiber, and a camera.

21. A method of displaying an estimated location of an instrument, comprising:

determining a first location of the instrument based on first location data generated by a set of one or more location sensors for the instrument, the first location data corresponding to a first time period;

after the first time period, receiving a user command to move the instrument during a second time period;

estimating a second location of the instrument based on the first location and the received user command, the estimated second location corresponding to the second time period;

confirming the estimated second location based on second location data generated by the set of location sensors, the second location data corresponding to the second time period; and

causing the estimated second location to be displayed prior to the confirmation of the estimated second location.

22. The method of claim 21, wherein the difference between the first time period and the second time period is based on a processing period for the confirmation of the second location estimate.

23. The method of claim 21, wherein:

the estimating of the second location has a first processing delay;

the confirming of the estimated second location has a second processing delay; and

the first processing delay is less than the second processing delay.

24. The method of claim 21, further comprising:

estimating the first location of the instrument for the first time period; and

confirming the estimated first location based on the first location data to determine the first location.

25. The method of claim 21, further comprising:

maintaining a set of predictive states for a predicted location of the instrument, each of the predictive states associated with a corresponding probability; and

estimating the second location based on the predictive state having the highest probability for the second time period.

26. The method of claim 25, further comprising:

adjusting a speed of movement of the instrument in response to the probability of the predictive state being less than a threshold probability.

27. The method of claim 21, further comprising:

receiving second location data from set of the location sensors during the second time period, wherein the confirming of the estimated second location comprises determining the second location of the instrument based on the second location data; and

causing the estimated second location to be displayed prior to determining the second location based on the second location data.

28. The method of claim 27, further comprising:

comparing the second location to the estimated second location; and

storing the second location in the memory in response to the second location being different from the estimated second location.

29. The method of claim 28, further comprising:

estimating a third location of the instrument based on the estimated second location, the estimated third location corresponding to a third time period after the second time period; and

updating the estimated third location based on the second location in response to the second location being different from the estimated second location.

30. The method of claim 21, wherein the set of location sensors comprises at least one of an electromagnetic (EM) sensor, a shape-sensing fiber, and a camera.