Surgical system user interface using cooperatively-controlled robot

• US 9,815,206 B2
• Filed: 09/25/2014
• Issued: 11/14/2017
• Est. Priority Date: 09/25/2014
• Status: Active Grant

• Current Assignee: Johns Hopkins University
• Original Assignee: Johns Hopkins University
• Inventors: Balicki, Marcin A., Kazanzides, Peter, Deguet, Anton, Taylor, Russell H.
• Primary Examiner(s): Amin, Bhavesh V
• Application Number:
  US14/497,178
• Publication Number:
  US 20160089212A1
• Time in Patent Office: 1,146 Days
• Field of Search: 700245-264
• CPC Class Codes: A61B 2034/258 providing specific settings for specific users
• CPC Class Codes: A61B 2090/0807 Indication means
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/77 Manipulators with motion or force scaling
• CPC Class Codes: A61B 90/20 Surgical microscopes characterised by non-optical aspects
• CPC Class Codes: A61B 90/30 Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 13/003 by means of an audio-responsive input audible safety signals B25J19/061
• CPC Class Codes: B25J 13/02 Hand grip control means handles or pedals for crane control B66C13/56; for measuring the force applied to control members G01L5/22; hand-held casings for switching devices, e.g. joy-sticks H01H9/0214
• CPC Class Codes: B25J 13/04 Foot-operated control means
• CPC Class Codes: B25J 13/06 Control stands, e.g. consoles, switchboards
• CPC Class Codes: B25J 13/065 comprising joy-sticks
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/67 : for remote operation
• CPC Class Codes: Y10S 901/06 : Communication with another machine

Abstract

According to some embodiments of the present invention, a cooperatively controlled robot includes a robotic actuator assembly comprising a tool holder and a force sensor, a control system adapted to communicate with the robotic actuator assembly and the force sensor, and an output system in communication with the control system. The tool holder is configured to receive a tool to be manipulated by a user. The control system is configured to receive an instruction from a user to switch from a robot control mode into a user interface control mode. The force sensor is configured to measure at least one of a force and a torque applied to the tool, and the control system is configured to receive an indication of the at least one of a force and a torque applied to the tool and manipulate the output system based on the indication.

Claims

1. A cooperatively controlled robot, comprising:

a robotic actuator assembly comprising a tool holder and a force sensor;

a control system adapted to communicate with said robotic actuator assembly and said force sensor; and

an output system in communication with said control system,

wherein said tool holder is configured to receive a tool to be manipulated by a user,

wherein said control system is configured to receive an instruction from a user to switch from a robot control mode into a user interface control mode,

wherein said force sensor is configured to measure at least one of a force and a torque applied to said tool by said user, and

wherein said control system, in said user interface control mode, is configured to receive an indication of said at least one of a force and a torque applied to said tool by said user and manipulate said output system based on said indication while disabling motion of said robot.

2. A cooperatively controlled robot according to claim 1, further comprising:

a microphone in communication with said control system, wherein said instruction from a user to switch from a robot control mode into a user interface control mode comprises a verbal signal detected by said microphone.

3. A cooperatively controlled robot according to claim 1, further comprising:

a pedal in communication with said control system, wherein said instruction from a user to switch from a robot control mode into a user interface control mode comprises a push or a release of said pedal.

4. A cooperatively controlled robot according to claim 1, further comprising:

a physical button in communication with said control system, wherein said instruction from a user to switch from a robot control mode into a user interface control mode comprises a command received through said physical button.

5. A cooperatively controlled robot according to claim 1, further comprising:

a second output system in communication with said control system, wherein said instruction from a user to switch from a robot control mode into a user interface control mode comprises a command received through a user interface of said second output system.

6. A cooperatively controlled robot according to claim 1, wherein said output system is an audio system.

7. A cooperatively controlled robot according to claim 1, wherein said output system is a visual display system.

8. A cooperatively controlled robot according to claim 7, wherein said control system is configured to manipulate said visual display system by changing a position of a cursor based on said indication.

9. A cooperatively controlled robot according to claim 7, wherein said control system is configured to manipulate said visual display system by changing a velocity of a cursor based on said indication.

10. A cooperatively controlled robot according to claim 7, wherein said visual display system is configured to display an image, and wherein said control system is configured to manipulate said image based on said indication.

11. A cooperatively controlled robot according to claim 10, wherein said image is created using data from a patient.

12. A cooperatively controlled robot according to claim 7, wherein said visual display system is configured to display an image, and wherein said control system is configured to manipulate said visual display system by zooming into or out of said image based on said indication.

13. A cooperatively controlled robot according to claim 1, wherein said output system is a second robotic actuator assembly.

14. A cooperatively controlled robot according to claim 1, wherein said output system is a microscope.

15. A cooperatively controlled robot according to claim 1, wherein said output system is a light source.

16. A cooperatively controlled robot according to claim 1, wherein said control system is configured to manipulate said output system by browsing through a list of parameters and selecting a parameter based on said indication.

17. A cooperatively controlled robot according to claim 1, wherein said control system is configured to perform a button press function on a condition that said force is exerted along a predetermined axis and exceeds at least one of a predetermined duration and magnitude.

18. A cooperatively controlled robot according to claim 1, wherein said tool is a surgical instrument.

19. A cooperatively controlled robot according to claim 1, wherein said tool is a joystick tool.

20. A surgical system user interface for a cooperatively controlled robot, comprising:

a control system adapted to communicate with a robotic actuator assembly and a force sensor; and

an output system in communication with said control system,

wherein said control system is configured to receive an instruction from a user to switch from a robot control mode into a user interface control mode, and

wherein said control system, in said user interface control mode, is configured to receive from a force sensor an indication of at least one of a force and a torque applied to a tool by a user and manipulate said output system based on said indication while disabling motion of said robot.