CONTEXT AWARE SURGICAL SYSTEMS

• US 20160015471A1
• Filed: 03/14/2015
• Published: 01/21/2016
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Synaptive Medical (Barbados) Inc.
• Original Assignee: Synaptive Medical (Barbados) Inc.
• Inventors: Piron, Cameron, Wood, Michael, Sela, Gal, Richmond, Joshua, Yuwaraj, Murugathas, McFadyen, Stephen, Panther, Alex, Shanmugaratnam, Nishanthan, Lau, William, Thomas, Monroe M., Hodges, Wes, Alexander, Simon, Gallop, David
• Granted Patent: US 9,788,906 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 1/00059 provided with identification means for the endoscope
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 1/0655 Control therefor
• CPC Class Codes: A61B 18/14 Probes or electrodes therefor
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/252 indicating steps of a surgical procedure
• CPC Class Codes: A61B 2034/256 having a database of accessory information, e.g. including context sensitive help or scientific articles
• CPC Class Codes: A61B 2090/306 using optical fibres
• CPC Class Codes: A61B 2090/3618 with a mirror
• CPC Class Codes: A61B 2090/373 using light, e.g. by using optical scanners
• CPC Class Codes: A61B 2090/3735 Optical coherence tomography [OCT]
• CPC Class Codes: A61B 2090/374 NMR or MRI
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 2090/378 using ultrasound
• CPC Class Codes: A61B 2090/3983 Reference marker arrangements for use with image guided surgery
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 6/03 : Computed tomography [CT] echo-tomography A61B8/14
• CPC Class Codes: A61B 6/5247 : combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
• CPC Class Codes: A61B 8/0808 : for diagnosis of the brain
• CPC Class Codes: A61B 8/0841 : for locating instruments
• CPC Class Codes: A61B 8/13 : Tomography A61B8/10, A61B8/12 take precedence; tomography for radiation diagnosis A61B6/02
• CPC Class Codes: A61B 90/30 : Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: A61B 90/50 : Supports for surgical instruments, e.g. articulated arms
• CPC Class Codes: A61B 90/90 : Identification means for patients or instruments, e.g. tags
• CPC Class Codes: A61B 90/94 : coded with symbols, e.g. text
• CPC Class Codes: A61B 90/96 : using barcodes
• CPC Class Codes: A61B 90/98 : using electromagnetic means, e.g. transponders
• CPC Class Codes: G06T 11/003 : Reconstruction from projections, e.g. tomography
• CPC Class Codes: G06T 2207/10081 : Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/10088 : Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/10104 : Positron emission tomography [PET]
• CPC Class Codes: G06T 2207/10132 : Ultrasound image
• CPC Class Codes: G06T 2207/30016 : Brain
• CPC Class Codes: G06T 7/0012 : Biomedical image inspection
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

Systems and methods are provided in which devices that are employed during a medical procedure are adaptively configured during the medical procedure, based on input or feedback that is associated with the current state, phase or context of the medical procedure. In some example embodiments, the input is obtained via the identification of one or more medical instruments present within a region of interest, and this input may be employed to determine configuration parameters for configuring the device. In other example embodiments, the input may be based on the image-based detection of a measure associated with the phase or context of the medical procedure, and this input may be employed to adaptively control the device based on the inferred context or phase of the medical procedure. In other embodiments, images from one imaging modality may be employed to adaptively switch to another imaging modality.

Claims

1. A computer implemented method of adaptively and intraoperatively configuring a device used during a medical procedure, the method comprising:

identifying a medical instrument during the medical procedure;

obtaining one or more customized configuration parameters for adaptively configuring the device during the medical procedure, where the customized configuration parameters are selected based on the identity of the medical instrument; and

configuring the device according to the customized configuration parameters.

2. The method according to claim 1 wherein the medical instrument is automatically identified.

3. The method according to claim 2 wherein the medical instrument is identified by:

detecting a signal from one or more fiducial markers associated with the medical instrument; and

processing the signal to identify the medical instrument.

4. The method according to claim 3 wherein the one or more fiducial markers are selected from the group consisting of passive markers, active markers, glyphs and RFID tags.

5. The method according to claim 3 wherein the medical instrument is identified based on image analysis employing a known shape of the medical instrument.

6. The method according to claim 1 wherein identifying one or more medical devices comprises receiving input from an operator, the input identifying the medical instrument.

7. The method according to any one of claims 1 to 6 wherein identifying the medical instrument comprises identifying the type of the medical instrument.

8. The method according to any one of claims 1 to 6 wherein identifying the medical instrument comprises uniquely identifying the medical instrument.

9. The method according to claim 8 wherein uniquely identified medical instrument is associated with one or more operators.

10. The method according to claim 8 wherein uniquely identified medical instrument associated is associated with one or more regions of use.

11. The method according to any one of claims 1 to 10 further comprising:

receiving input identifying one or more operators present during the medical procedure;

wherein the customized configuration parameters are further associated with the identity of the one or more operators.

12. The method according to any one of claims 1 to 10 further comprising:

receiving input identifying the medical procedure;

wherein the customized configuration parameters are further associated with the medical procedure.

13. The method according to any one of claims 1 to 12 further comprising:

determining a current stage of the medical procedure;

wherein the customized configuration parameters are further associated with the current stage of the medical procedure.

14. The method according to any one of claims 1 to 13 wherein the customized configuration parameters are obtained from a data storage device comprising customized configuration parameters for adaptively configuring a plurality of devices during the medical procedure, the method further comprising:

identifying one or more additional medical instruments employed concurrently during the medical procedure;

wherein the one or more customized configuration parameters are selected based on the identity of identified medical instruments.

15. The method according to claim 14 wherein the customized configuration parameters stored in the data storage device are ranked based on the identity of different medical instruments, and wherein the customized configuration parameters are obtained for the identified instrument having the highest ranking.

16. The method according to any one of claims 1 to 15 wherein the device is an auxiliary device.

17. The method according to any one of claims 1 to 16 wherein the device is an illumination device.

18. The method according to any one of claims 1 to 16 wherein the device is computer hardware for generating a user interface on a display.

19. The method according to any one of claims 1 to 16 wherein the device is a robotic positioning device.

20. The method according to any one of claims 1 to 16 wherein the device is an imaging device.

21. The method according to claim 20 wherein the customized configuration parameters comprise one or more of colour balance, brightness, depth of field, magnification, field of view, working distance, and illumination conditions.

22. The method according to claim 20 wherein the imaging device is a multimodal imaging device, and wherein the customized configuration parameters comprise a selection of an imaging modality.

23. The method according to any one of claims 1 to 22 wherein the medical procedure is a surgical procedure.

24. A system for adaptively and intraoperatively configuring a device used during a medical procedure, comprising:

a data storage device comprising customized configuration parameters for adaptively configuring one or more devices during the medical procedure;

a control and processing system interfaced with the device and the data storage device, said control and processing system comprising one or more processors and memory coupled to said one or more processors, said memory storing instructions, which, when executed by said one or more processors, causes said one or more processors to perform operations comprising;

identifying a medical instrument during the medical procedure;

obtaining, from the data storage device, one or more customized configuration parameters for adaptively configuring the device during the medical procedure, where the customized configuration parameters are customized based on the identity of the medical instrument; and

configuring the device according to the customized configuration parameters.

25. A computer implemented method of adaptively configuring a device used during a medical procedure, the method comprising:

obtaining one or more images of a region of interest associated with the medical procedure;

processing the one or more images to identify a context measure associated with the current state of the medical procedure;

obtaining one or more customized configuration parameters for adaptively configuring the device during the medical procedure, where the customized configuration parameters are customized based on the context measure; and

configuring the device according to the customized configuration parameters.

26. The method according to claim 25 wherein the context measure is associated with a current phase of the medical procedure.

27. The method according to claim 25 wherein the context measure is associated with the type of medical procedure.

28. The method according to claim 25 wherein the context measure is associated with the presence of one or more tissue types within the region of interest.

29. The method according to claim 25 wherein the context measure is associated with the presence of one or more fluids within the region of interest.

30. The method according to claim 25 wherein one of the one or more fluids is a biological fluid.

31. The method according to claim 25 wherein the biological fluid is blood.

32. The method according to claim 25 wherein the context measure is associated with the presence of one or more medical instruments within the region of interest.

33. The method according to claim 26 wherein processing the one or more images to identify the context measure associated with the current phase of the medical procedure comprises:

acquiring hyperspectral image data from a region of interest;

processing hyperspectral image data to identify one or more regions with similar spectral response among pixels;

comparing an average spectral response from each region to reference spectra associated with different phases of the medical procedure; and

identifying the phase of medical procedure based on similarity between average spectral response and reference spectra.

34. The method according to claim 26 wherein processing the one or more images to identify the current phase of the medical procedure comprises:

processing the one or more images to identify one or more tissue types present within the region of interest; and

determining the phase of the medical procedure based on the one or more tissue types.

35. The method according to claim 26 wherein processing the one or more images to identify the current phase of the medical procedure comprises:

performing image analysis to identify one or more medical instruments based on the known shape of one or more medical instrument; and

determining the phase of the medical procedure based on the presence of the one or more medical instruments.

36. The method according to claim 25 wherein the one or more images are optical images.

37. The method according to claim 36 wherein at least one of the optical images are based on detection in at least the visible spectrum.

38. The method according to claim 36 wherein at least one of the optical images are based on detection in at least the infrared spectrum.

39. The method according to claim 25 wherein at least one of the images is an optical coherence tomography image.

40. The method according to claim 25 wherein the images are obtained, at least in part, by magnetic resonance imaging.

41. The method according to claim 25 wherein the images are obtained, at least in part, by ultrasound imaging.

42. The method according to any one of claims 25 to 41 wherein processing the images comprises processing two or more images from different imaging modalities.

43. The method according to any one of claims 25 to 42 wherein the device is a surgical tool.

44. The method according to any one of claims 25 to 42 wherein the device is an auxiliary device.

45. The method according to any one of claims 25 to 42 wherein the device is an illumination device.

46. The method according to according to any one of claims 25 to 42 wherein the device comprises computer hardware for generating a user interface on a display.

47. The method according to according to any one of claims 25 to 42 wherein the device comprises a robotic positioning device.

48. The method according to according to any one of claims 25 to 42 wherein the device is an imaging device.

49. The method according to any one of claims 25 to 48 wherein controlling the device comprises:

obtaining one or more customized configuration parameters for adaptively configuring the device during the medical procedure, where the customized configuration parameters are customized based on the current phase of the medical procedure; and

configuring the device according to the customized configuration parameters.

50. The method according to claim 49 wherein the device is an imaging device, and wherein the customized configuration parameters comprise one or more of colour balance, brightness, depth of field, magnification, field of view, working distance, and illumination conditions.

51. The method according to claim 49 wherein the imaging device is a multimodal imaging device, and wherein the customized configuration parameters comprise a selection of an imaging modality.

52. The method according to any one of claims 25 to 51 wherein the medical procedure is a surgical procedure.

53. A system for adaptively and intraoperatively configuring a device used during a medical procedure, comprising:

a data storage device comprising customized configuration parameters for adaptively configuring one or more devices during the medical procedure;

a control and processing system interfaced with the device and the data storage device, said control and processing system comprising one or more processors and memory coupled to said one or more processors, said memory storing instructions, which, when executed by said one or more processors, causes said one or more processors to perform operations comprising;

obtaining one or more images of a region of interest associated with the medical procedure;

processing the one or more images to identify a context measure associated with the current state of the medical procedure;

obtaining one or more customized configuration parameters for adaptively configuring the device during the medical procedure, where the customized configuration parameters are customized based on the context measure; and

configuring the device according to the customized configuration parameters.

54. A computer implemented method of adaptively controlling a first imaging modality and a second imaging modality during a medical procedure, the method comprising:

while obtaining first images with the first imaging modality, intermittently obtaining one or more second images with the second imaging modality;

processing the second images to calculate, for a plurality of regions within the second images, an image measure associated with the second imaging modality; and

in the event that the image measure for one or more regions is within a pre-selected range, increasing the rate of acquisition of the second images.

55. The method according to claim 54 further comprising, in the event that the image measure for one or more regions is within a pre-selected range, continuously acquiring the second images.

56. The method according to claim 54 further comprising, in the event that the image measure for one or more regions is within a pre-selected range, reducing the rate of acquisition of the first images.

57. The method according to claim 54 further comprising, in the event that the image measure for one or more regions is within a pre-selected range, terminating acquisition of the first images.

58. The method according to any one of claims 54 to 57 wherein the second images are intermittently obtained at a pre-selected rate.

59. The method according to any one of claims 54 to 57 wherein an illumination source associated with the first imaging modality is turned off while obtaining the second images with the second imaging modality.

60. The method according to any one of claims 54 to 59 wherein the image measure is associated with the presence of one or more tissue types identifiable with the second imaging modality.

61. The method according to claim 60 wherein the tissue type comprise one of tumor tissue and white matter.

62. The method according to any one of claims 54 to 61 wherein the second imaging modality comprises one of fluorescence imaging, Raman imaging, hyperspectral imaging, optical coherence tomography imaging, and polarization-sensitive optical imaging.

63. The method according to any one of claims 54 to 61 wherein the second imaging modality provides hyperspectral images, wherein the method of processing the second images comprises:

processing the second images to calculate, for a plurality of regions within the second images, an image measure associated with the presence of a pre-selected spectral signature.

64. The method according to any one of claims 54 to 63 wherein the first imaging modality employs optical imaging in the visible spectrum.

65. The method according to any one of claims 54 to 64 further comprising overlaying the portion of the second images corresponding to the one or more regions with the first images.

66. The method according to any one of claims 54 to 64 further comprising displaying both the first images and the second images.

67. A system for adaptively controlling one or more imaging devices during a medical procedure, comprising:

a control and processing system interfaced with the one or more imaging devices, said control and processing system comprising one or more processors and memory coupled to said one or more processors, said memory storing instructions, which, when executed by said one or more processors, causes said one or more processors to perform operations comprising;

obtaining first images with a first imaging modality and intermittently obtaining one or more second images with a second imaging modality;

processing the second images to calculate, for a plurality of regions within the second images, an image measure associated with the second imaging modality; and

in the event that the image measure for one or more regions is within a pre-selected range, increasing the rate of acquisition of the second images.

68. A computer implemented method of adaptively controlling one or more imaging devices during a medical procedure, the method comprising:

a) obtaining one or more first images with a first imaging modality;

b) processing the first images to calculate, for a plurality of regions within the first images, an image measure associated with the suitability of a second imaging modality;

c) in the event that the image measure for one or more regions lies within a pre-selected range, acquiring one or more second images with the second imaging modality.

69. The method according to claim 68 wherein the first imaging modality employs non-polarized optical imaging, and the second imaging modality employs cross-polarized optical imaging, and wherein the image measure is associated with glare.

70. The method according to claim 69 wherein the image measure is calculated by determining the relative intensity of the regions, and wherein the pre-selected range corresponds to intensity values indicative of glare.

71. The method according to claim 68 wherein the image measure is associated with a spectral signature known to be suitable for imaging with the second imaging modality.

72. The method according to claim 71 wherein the first imaging modality employs in the visible spectrum, and the second imaging modality employs near infrared imaging, and wherein the image measure is associated with a spectral signature known to be suitable for near infrared imaging.

73. The method according to claim 72 further wherein the spectral signature is associated with a substance or tissue type.

74. The method according to claim 72 further wherein the spectral signature is associated with a substance or tissue type or substance that is transparent to near infrared imaging.

75. The method according to claim 74 further wherein the substance is blood.

76. The method according to claim 72 further wherein the first imaging modality employs hyperspectral imaging.

77. The method according to claim 72 further wherein the first imaging modality employs colour imaging, and wherein the spectral signature determined based on the relative intensity of different image colours.

78. The method according to any one of claims 72 to 77 wherein the image measure is calculated by determining the relative image intensity within a pre-selected spectral region.

79. The method according to any one of claims 68 to 78 further comprising repeating steps a) to c) one or more times.

80. The method according to any one of claims 68 to 78 wherein step c) further comprises continuously acquiring additional second images with the second imaging modality.

81. The method according to any one of claims 68 to 80 further comprising overlaying the portion of the second images corresponding to the one or more regions with the first images.

82. The method according to any one of claims 68 to 80 further comprising displaying both the first images and the second images.

83. A system for adaptively controlling a one or more imaging devices during a medical procedure, comprising:

a control and processing system interfaced with the one or more imaging devices, said control and processing system comprising one or more processors and memory coupled to said one or more processors, said memory storing instructions, which, when executed by said one or more processors, causes said one or more processors to perform operations comprising;

obtaining one or more first images with a first imaging modality;

processing the first images to calculate, for a plurality of regions within the first images, an image measure associated with the suitability of a second imaging modality;

in the event that the image measure for one or more regions lies within a pre-selected range, acquiring one or more second images with the second imaging modality.

84. A method of performing adaptive illumination while performing optical imaging during a medical procedure, the method comprising:

determining the field of view of an optical imaging device employed during the medical procedure;

determining configuration parameters of an illumination source for improving the homogeneity of illumination within the field of view;

configuring the illumination source according to the configuration parameters.

85. The method according to claim 84 wherein the configuration parameters are determined by:

a) obtaining an image with the imaging device;

b) processing the image to determine an image measure associated with the homogeneity of illumination;

c) varying the position and/or orientation of the illumination source; and

d) repeating steps (a)-(c) until the image measure lies within a pre-selected range.

86. The method according to claim 84 wherein the configuration parameters are determined by:

a) determining the current position and orientation of the illumination source relative to the field of view;

b) calculating, based on a model of the illumination source, an image measure associated with homogeneity of illumination within the field of view;

c) varying the position and/or orientation of the illumination source within the model; and

d) repeating steps b)-c) until the image measure lies within a pre-selected range.

87. The method according to claim 86 wherein the model of the illumination source accounts for reflections or scattering from one or more surfaces into the field of view.

88. The method according to claim 86 wherein the model of the illumination source is a non-sequential ray tracing model.

89. A system for performing adaptive illumination while performing optical imaging during a medical procedure, comprising:

a control and processing system interfaced with the optical imaging device and the illumination source, said control and processing system comprising one or more processors and memory coupled to said one or more processors, said memory storing instructions, which, when executed by said one or more processors, causes said one or more processors to perform operations comprising;

determining the field of view of an optical imaging device employed during the medical procedure;

determining configuration parameters of an illumination source for improving the homogeneity of illumination within the field of view;

configuring the illumination source according to the configuration parameters.