Motion and video capture for tracking and evaluating robotic surgery and associated systems and methods

• US 9,026,247 B2
• Filed: 03/29/2012
• Issued: 05/05/2015
• Est. Priority Date: 03/30/2011
• Status: Active Grant

• Current Assignee: University of Washington
• Original Assignee: University of Washington
• Inventors: White, Lee Woodruff, Kowalewski, Timothy Mariusz, Lendvay, Thomas S., Hannaford, Blake
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): Figueroa, Jaime
• Application Number:
  US13/434,585
• Publication Number:
  US 20120253360A1
• Time in Patent Office: 1,132 Days
• Field of Search: 700/245, 700/250, 700/253, 700/254, 700/258, 700/259, 318/568.11, 318/568.12, 318/568.13, 318/568.16, 318/568.2, 318/568.21
• US Class Current: 700/245
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/35 for telesurgery

Abstract

The present technology is directed to motion and video capture for tracking and evaluating robotic surgery. In one embodiment, the system includes at least one tracking device coupled to a remote surgical tool. The tracking device is configured to use one or more sensors to sense one or more physical variables such as movement and electrical contact. In some embodiments, the data from multiple individual sensors is synchronized, received, and stored by a digital information system. The digital information system is configured to analyze the data to objectively assess surgical skill.

Claims

1. A system for tracking performance of a remotely controlled tool, the system comprising:

a modification device coupled to the remotely controlled tool, wherein the modification device comprises a motion capture device having sensors at a first location configured to sense one or more physical variables associated with use of the remotely controlled tool, the physical variables including position and orientation data of at least one joint of the remotely controlled tool at a second location away from the first location, and further wherein the modification device uses multiple recordings to sense the one or more physical variables;

a data transmission component operably coupled to the modification device and configured to transmit data, wherein the data includes information associated with the one or more physical variables;

a synchronization component configured to synchronize the multiple recordings; and

a digital information system configured to receive and store the transmitted data.

2. The system of claim 1 wherein:

the at least one joint includes at least one rotating spindle;

the remotely controlled tool comprises an end effector configured to perform a surgical function; and

the end effector is driven by the at least one rotating spindle.

3. The system of claim 2 wherein the modification device is configured to sense rotation of the at least one spindle.

4. The system of claim 3 wherein the modification device is configured to sense rotation of the at least one spindle with at least one of potentiometers, optical encoders, Hall effect sensors, or a combination thereof.

5. The system of claim 2 wherein the digital information system is configured to compute motion of the end effector.

6. The system of claim 1 wherein the motion capture device is configured to transmit a signal corresponding to the position and orientation of the remotely controlled tool with respect to a reference point.

7. The system of claim 1 wherein the modification device includes a digital device configured to broadcast a unique serial number associated with the remotely controlled tool.

8. The system of claim 1 wherein the modification device comprises a video capture device configured to record video being output to a user, and wherein the digital information system is configured to receive and store the recorded video.

9. The system of claim 1 wherein the modification device comprises an electrical contact sensor configured to transmit a signal relating to at least one of temperature, pH, immunohistochemistry, electrical impedance, physical impedance, an optical characteristic, tissue contact, transmissibility of radiation, or an electromagnetic characteristic.

20. The system of claim 7 wherein the digital information system is configured to reference calibration parameters specific to the remotely controlled tool based, at least in part, on a unique serial number associated with the remotely controlled tool.

21. The system of claim 1 wherein the digital information system is configured to analyze and determine position and orientation of components of the remotely controlled tool using a combination of recordings from at least one potentiometer and at least one electromagnetic tracker of the modification device.

22. The system of claim 1 wherein the digital information system is configured to simultaneously record and synchronize data of the remotely controlled tool and physiological data associated with a user of the tracking system.

23. The system of claim 1, further comprising a potentiometer calibration system configured to fix the remotely controlled tool in a baseline orientation corresponding with at least one recess of a calibration plate, and wherein:

the calibration plate is configured to interface with a proximal portion of the remotely controlled tool, and

the proximal portion of the remotely controlled tool includes at least one spindle comprising at least one protrusion configured to align with the at least one recess of the calibration plate.

24. The system of claim 1, further comprising an electromagnetic calibration system configured to determine the location of a tip of the remotely controlled tool with respect to a reference point, wherein the electromagnetic calibration system comprises a tip registration sleeve defining an internal lumen that corresponds with an outside diameter of the remotely controlled surgical tool.

10. A system for tracking performance of a robotic surgical tool, the system comprising:

a surgical tracker configured to be removably coupled to the surgical tool at a first location, the surgical tracker comprising a video capture device configured to capture video of the surgical tool and a motion capture device configured to sense the position of at least one joint of the surgical tool at a second location spaced apart from the location;

a data transmission component configured to transmit video and position data captured by the video capture device and the motion capture device, respectively; and

a digital information system configured to receive and store the transmitted video and position data, wherein the digital information system comprises a data recorder, a synchronization component, and a data analyzer.

11. The system of claim 10 wherein:

the at least one joint includes at least one rotating spindle;

the robotic surgical tool comprises an end effector;

the end effector is driven by the at least one rotating spindle; and

the motion capture device is configured to sense rotation of the at least one spindle.

12. The system of claim 11 wherein the end effector is driven by a plurality of rotating spindles, and wherein the motion capture device comprises a plurality of potentiometers, each potentiometer configured to sense rotation of an individual spindle.

13. The system of claim 10 wherein the motion capture device comprises an electromagnetic tracker configured to transmit a signal corresponding to the position and orientation of the robotic surgical tool with respect to a static reference point.

14. The system of claim 10 wherein the surgical tracker comprises a digital device configured to broadcast a unique serial number associated with the robotic surgical tool.

15. The system of claim 10 wherein the synchronization component is configured to time-sync data provided by the video capture device with data provided by the motion capture device.

16. The system of claim 10 wherein the synchronization component is configured to time-sync data provided by the video capture device and data provided by the motion capture device with data from an additional source.

17. The system of claim 10 wherein the surgical tracker comprises an electrical contact sensor configured to transmit a signal relating to at least one of temperature, pH, immunohistochemistry, electrical impedance, physical impedance, an optical characteristic, tissue contact, transmissibility of radiation, an electromagnetic characteristic.

18. The system of claim 10 wherein the data analyzer is configured to compute a performance indicator based on at least one of the quantity, direction, duration, pattern, force, path length, or sequence of movement of the robotic surgical tool.

19. The system of claim 10, further comprising:

a plurality of potentiometers, each potentiometer configured to sense rotation of an individual joint driving the robotic surgical tool, wherein the joint includes at least one rotating spindle;

an electromagnetic tracker configured to transmit a signal corresponding to the position and orientation of the robotic surgical tool with respect to a static reference point;

a digital device configured to broadcast a unique serial number associated with the robotic surgical tool;

an electrical contact sensor configured to transmit a signal relating to at least one of temperature, pH, immunohistochemistry, electrical impedance, physical impedance, an optical characteristic, tissue contact, transmissibility of radiation, an electromagnetic characteristic; and

an output device configured to output a performance indicator based on at least one of the quantity, direction, duration, pattern, force, path length, or sequence of movement of the robotic surgical tool.