System and method for enhanced data analysis with video enabled software tools for medical environments

• US 10,433,917 B2
• Filed: 07/17/2017
• Issued: 10/08/2019
• Est. Priority Date: 05/29/2009
• Status: Active Grant

• Current Assignee: Jack Wade
• Original Assignee: Jack Wade
• Inventors: Wade, Jack
• Primary Examiner(s): Nakhjavan, Shervin K
• Application Number:
  US15/652,031
• Publication Number:
  US 20190008593A1
• Time in Patent Office: 813 Days
• Field of Search: None
• CPC Class Codes: A61B 1/00006 of control signals
• CPC Class Codes: A61B 1/000096 using artificial intelligence
• CPC Class Codes: A61B 1/00011 characterised by signal transmission
• CPC Class Codes: A61B 1/3132 for laparoscopy
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing

Abstract

Medical software tools platform utilizes a surgical display to provide access to specific medical software tools, such as medically-oriented applications or widgets, that can assist those in the operating room, such as a surgeon or surgical team, with a surgery. In particular, an optical sensor located within an endoscopic camera may register the momentary change in the optical characteristics reflected from a tissue surface and in turn transmit the information to a medical image processing system which can also receive patient heart rate data and display relevant anomalies. Changes in various spectral components and the speed at which they change in relation to a source of stimulus (heartbeat, etc.) may indicate the arrival of blood, contrast agents or oxygen absorption. Combinations of these may indicate various states of differing disease or margins of tumors, and so forth. Also, changes in temperatures and the speed of change may indicate subsurface anomalies.

Claims

1. A system for medical software tools, comprising:

an image stream interface module configured to receive an image stream from a surgical camera;

a user interface overlay module configured to provide a user interface overlay adapted for presentation over the image stream;

an optical sensor located corresponding with said surgical camera for registering momentary changes in spectral characteristics reflected from a tissue surface under inspection;

a medical software tools module configured to provide a medical software tool through the user interface, the medical software tool being configured to perform an operation with respect to the image stream and provide an output adapted to be presented over the image stream; and

a medical image processing system for processing patient medical data and corresponding said patient medical data with said momentary changes in spectral characteristics for generating optical signature data indicative of various patient conditions.

2. The system of claim 1, wherein the user interface overlay module is further configured to present the output over the image stream.

3. The system of claim 1, wherein the surgical camera is an endoscope or a laparoscope.

4. The system of claim 1, wherein the image stream inter face module is further configured to receive the image stream from the surgical camera through an image stream processing system.

5. The system of claim 1, further comprising a medical device interface module configured to exchange data between a medical device and the medical software tool.

6. The system of claim 1, further comprising an image stream processing system interface module configured to exchange data between an image stream processing system and the medical software tool, the medical software tool being configured to modify a setting of the image stream processing system.

7. The system of claim 6, wherein the setting includes enabling or disabling application of an image stream processing algorithm to the image stream.

8. The system of claim 1, wherein the operation comprises disposing a visual tag over the image stream in association with an anatomical structure or tissue presented in the image stream, and the output comprises the visual tag.

9. The system of claim 8, wherein the operation further comprises associating a timer with respect to the visual tag.

10. The system of claim 1, the operation comprises measuring an anatomical structure or tissue presented in the image stream, and the output comprises a resulting measurement.

11. The system of claim 1, wherein the image stream comprises two-dimensional content, the operation comprises converting at least some of the two-dimensional content into three-dimensional content, and the output comprises the three-dimensional content.

12. The system of claim 1, wherein the operation comprises associating content in the image stream with a timer, and the output comprises a visual representation of the timer.

13. The system of claim 1, wherein the operation comprises identifying content in the image stream similar to reference content, and the output comprises a visual indication of the identified content.

14. The system of claim 1, wherein the operation comprises whereby all sets of said optical signature data change rates are compared with data indicative of a patient vital signs data, and wherein this data is analyzed so that any anomalies are detected and y detection displayed to the user.

15. The system of claim 14 wherein changes in one of more of said spectral components and said speed at which said spectral components change in response to a patient vitals source data stimulus indicative of blood flow can be used to characterize various types or stages of disease.

16. The system of claim 15 wherein margins of tumors or diseased areas may be detected based on comparisons between present said spectral component changes and prior said spectral component changes.

17. A method for medical software tools, comprising:

a computer system receiving an image stream from a surgical camera;

the computer system providing a user interface overlay adapted for presentation over the image stream;

the computer system providing a medical software tool through the user interface;

the computer system performing an operation of the medical software tool with respect to the image stream; and

the computer system providing an output of the operation for presentation over the image stream;

an optical sensor located corresponding with said surgical camera for registering momentary changes in spectral characteristics reflected from a tissue surface under inspection and for providing optical sensor data output to said computer system;

a medical software tools module configured to provide a medical software tool through the user interface, the medical software tool being configured to perform an operation with respect to said image stream and provide an output adapted to be presented over the image stream; and

a medical image processing system for processing patient medical data and corresponding said patient medical data with said momentary changes in spectral characteristics for generating optical signature data indicative of various patient conditions.

18. The method of claim 17, further comprising the computer system presenting the output over the image stream.

19. The method of claim 17, wherein the surgical camera is an endoscope or a laparoscope.

20. The method of claim 17, wherein the computer system receives the image stream from the surgical camera through an image stream processing system.

21. The method of claim 20, wherein the medical software tool is configured to modify a setting of the image stream processing system.

22. The method of claim 21, wherein the setting includes enabling or disabling application of an image stream processing algorithm to the image stream.

23. The method of claim 17, wherein the operation comprises disposing a visual tag over the image stream in association with an anatomical structure or tissue presented in the image stream, and the output comprises the visual tag.

24. The method of claim 23, wherein the operation further comprises associating a timer with respect to the visual tag.

25. The method of claim 17, wherein the operation comprises measuring an anatomical structure or a tissue presented in the image stream, and the output comprises a resulting measurement.

26. The method of claim 17, wherein the image stream comprises two-dimensional content, the operation comprises converting at least some of the two-dimensional content into three-dimensional content, and the output comprises the three-dimensional content.

27. The method of claim 17, wherein the operation comprises associating content in the image stream with a timer, and the output comprises a visual representation of the timer.

28. The method of claim 17, wherein the operation comprises identifying content in the image stream similar to reference content, and the output comprises a visual indication of the identified content.

29. The method of claim 17, wherein the operation comprises whereby all sets of said optical signature data change rates are compared with data indicative of a patient vital signs data, and wherein this data is analyzed so that any anomalies are detected and y detection displayed to the user.

30. The method of claim 29 wherein changes in one of more of said spectral components and said speed at which said spectral components change in response to a patient vitals source data stimulus indicative of blood flow can be used to characterize various types or stages of disease.

31. The method of claim 30 wherein margins of tumors or diseased areas may be detected based on comparisons between present said spectral component changes and prior said spectral component changes.

32. A computer program product embedded on non-transitory computer storage media, which when executed by a computer, causes the computer to implement a method for medical software tools, the computer program product comprising:

code for receiving an image stream from a surgical camera;

code for providing a user interface overlay adapted for presentation over the image stream;

code for providing a medical software tool through the user interface;

code for performing an operation of the medical software tool with respect to the image stream;

code for providing an output of an operation for presentation over the image stream; and

code for a medical image processing system for processing patient medical data and corresponding said patient medical data with momentary changes in spectral characteristics for generating optical signature data indicative of various patient conditions.

33. A method for medical software tools, comprising:

a computer system receiving an image stream from a surgical camera;

the computer system providing a user interface overlay adapted for presentation over the image stream;

the computer system providing a medical software tool through the user interface;

the computer system performing an operation of the medical software tool with respect to the image stream; and

the computer system providing an output of the operation for presentation over the image stream;

an optical sensor located corresponding with said surgical camera for registering momentary changes in spectral characteristics reflected from a tissue surface under inspection and for providing optical sensor data output to said computer system;

a medical software tools module configured to provide a medical software tool through the user interface, the medical software tool being configured to perform an operation with respect to said image stream and provide an output adapted to be presented over the image stream; and

a medical image processing system for measuring changes in color intensity and rates at which said color intensity changes occur in response to at least one of either a patient heart beat pushing blood, a patient lung pushing oxygen or detection of light from any one of a plurality of sources wherein said measured changes in color intensity are compared with measured changes in color intensity from surrounding tissue areas for detecting anomalies indicative of diseased or abnormal tissue.

34. The method of claim 33 including measuring a photoacoustic response corresponding with pressure changes occurring in response to various light sources heating up tissue under observation.

35. The method of claim 33 including detection of tumor margins identified by comparing color spectral changes and rates of color spectral change in real time in a diseased area with said color spectral changes and said rates of color spectral change occurring in adjacent or surrounding proximate areas where said areas are identified by finding sets of readings with similar spectral signatures or rates of change.

36. The method of claim 35 wherein said color spectral changes and said rates of color spectral changes are analyzed using artificial intelligence referencing that a database of historical samples.

37. A medical software tools system, comprising:

a computer system receiving an image stream from a surgical camera;

the computer system providing a user interface overlay adapted for presentation over the image stream;

the computer system further providing a medical software tool through the user interface;

the computer system further performing an operation of the medical software tool with respect to the image stream; and

the computer system providing an output of the operation for presentation over the image stream;

an optical sensor located corresponding with said surgical camera for registering momentary changes in spectral characteristics reflected from a tissue surface under inspection and for providing optical sensor data output to said computer system;

a medical software tools module configured to provide a medical software tool through the user interface, the medical software tool being configured to perform an operation with respect to said image stream and provide an output adapted to be presented over the image stream; and

a medical image processing system for measuring changes in color intensity and rates at which said color intensity changes occur in response to at least one of either a patient heart beat pushing blood, a patient lung pushing oxygen or detection of light from any one of a plurality of sources wherein said measured changes in color intensity are compared with measured changes in color intensity from surrounding tissue areas for detecting anomalies indicative of diseased or abnormal tissue.

38. The system of claim 37 including measuring a photo-acoustic response corresponding with pressure changes occurring in response to various light sources heating up tissue under observation.

39. The system of claim 37 including detection of tumor margins identified by comparing color spectral changes and rates of color spectral change in real time in a diseased area with said color spectral changes and said rates of color spectral change occurring in adjacent or surrounding proximate areas where said areas are identified by finding sets of readings with similar spectral signatures or rates of change.

40. The system of claim 39 wherein said color spectral changes and said rates of color spectral changes are analyzed using artificial intelligence referencing that a database of historical samples.