Robotized surgery system with improved control

• US 9,360,934 B2
• Filed: 04/01/2011
• Issued: 06/07/2016
• Est. Priority Date: 04/07/2010
• Status: Active Grant

• Current Assignee: Asensus Surgical Italia Srl (Asensus Surgical, Inc.)
• Original Assignee: Transenterix Italia S.R.L. (Asensus Surgical, Inc.)
• Inventors: Ruiz Morales, Emilio, Brasset, Damien, Invernizzi, Paolo
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): Nguyen, Robert
• Application Number:
  US13/639,809
• Publication Number:
  US 20130030571A1
• Time in Patent Office: 1,894 Days
• Field of Search: 700/259
• US Class Current: 1/1
• CPC Class Codes: A61B 17/00 Surgical instruments, devices or methods, e.g. tourniquets A61B18/00 takes precedence; contraceptive devices, pessaries, or applicators therefor A61F6/00; eye surgery A61F9/007; ear surgery A61F11/00
• CPC Class Codes: A61B 2017/00216 with eye tracking or head position tracking control
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A robotized surgery system (10) comprises at least one robot arm (11) which acts under the control of a control console (12) intended for the surgeon. The console (12) comprises an eye tracking system (21) for detecting the direction of the surgeon'"'"'s gaze and for entering commands depending on the directions of the gaze detected. The console (22) comprises advantageously a screen (23) with at least one zone (23) for viewing the operating field and, among the commands which can be performed depending on the gaze directions, there is advantageously an automatic command for enabling or disabling the movement of the robot arm (11) when a gaze direction which falls within or outside of said zone (23) of the screen is detected.

Claims

1. A method for controlling a robotized surgery system comprising robot arms which act under the control of manipulators of a control console comprising a screen and intended for a surgeon, comprising the steps of:

showing pictures of the robot arms on the screen;

detecting the direction of the surgeon'"'"'s gaze and

controlling functions of the surgery system depending on the gaze direction detected; and

when a new robot arm has to be assigned to one manipulator,

selecting a picture of the new robot arm by one'"'"'s gaze direction on the picture of the new arm shown on the screen and then moving the one'"'"'s gaze direction on the screen into a screen position showing the picture of the robot arm assigned to said one manipulator to command the system to move the selected picture as dragging by means of one'"'"'s gaze movement and assigning the new robot arm to said one manipulator, and wherein the assignment of the manipulators and the robot arms is performed by an eye tracking system.

2. The method according to claim 1, comprising the steps of:

showing pictures of the operating field on at least one zone of the screen;

preventing the movement of the robot arms when the gaze direction detected does not fall within the screen or at least said screen zone and/or enabling the movement of the at least one robot arm when the direction of the gaze detected falls within the screen or at least said screen zone.

3. The method according to claim 1, comprising the steps of:

displaying on the screen selection areas associated with commands for the surgery system and, when a gaze direction which falls within one of said selection areas is detected, selecting the command associated with this area and.

4. The method according to claim 1, comprising the steps of:

showing on the screen a picture of the operating field, detecting the distance of the surgeon'"'"'s eyes from the screen and varying the enlargement and/or position of the picture depending on the distance detected and/or signalling whether the distance detected is within a predefined working space of the distance detection system and/or whether it is within a predefined viewing space of the picture of the operating field.

5. The method according to claim 1, comprising the steps of:

showing on the screen a picture of the operating field recorded by a telecamera, moved by a robot arm, and moving the robot arm;

for varying the viewing frame of the telecamera depending on the detected direction of the gaze on the screen picture.

6. The method of claim 2, wherein a further manual command confirming enabling is sent.

7. The method of claim 3, wherein the command selected is also confirmed by means of operation of a further manual confirmation device.

8. The method of claim 4, wherein the picture of the operating field is displayed in 3D.

9. A robotized surgery system comprising robot arms that act under the control of a control console intended for a surgeon,

wherein the console comprises an eye tracking system for detecting the direction of the surgeon'"'"'s gaze and for entering commands depending on the direction of the gaze detected,

wherein the console comprises a screen with at least one zone for viewing the operating field and a human machine interface that displays selection areas on the screen that are associated with commands,

wherein the tracking system estimates the direction of the surgeon'"'"'s gaze and performs the selection of commands associated with one area of the selection areas when it detects a gaze direction that falls within said one area,

wherein the console comprises two manipulators for operating the robot arms by two hands and the commands associated with the selection areas comprise commands for mutual assignment of manipulators and robot arms, and

wherein the assignment of the manipulators and the robot arms is performed by the eye tracking system.

10. The system according to claim 9, characterized in that the console comprises a screen with at least one zone for viewing the operating field and in that the tracking system generates a command for disabling the movement of at least one robot arm when a gaze direction which falls outside the screen or at least said zone of the screen is detected.

11. The system according to claim 9, characterized in that the screen forms part of a system for viewing the operating field in 3D.

12. The system according to claim 9, characterized in that the eye tracking system comprises at least one telecamera for recording the picture of at least the surgeon'"'"'s eyes and means for calculating the direction of the gaze depending on the picture taken.

13. The system according to claim 9, characterized in that it comprises a system for detecting the distance between the surgeon'"'"'s eyes and a screen showing a picture of the operating field, this distance detection system performing a variation in the enlargement and/or position of the picture of the operating field shown on the screen depending on a variation in the distance detected.

14. The system according to claim 9, characterized in that the console comprises a screen with at least one zone for viewing the operating field and in that the tracking system generates a command for enabling the movement of at least one robot arm when a gaze direction which falls inside the screen or at least said zone of the screen is detected, this command being combined with a manual command for confil ming enabling in order to activate movement of the robot arm.

15. The system according to claim 9, characterized in that the console comprises a screen with at least one zone for viewing the operating field recorded by a telecamera which is moved by a robot arm, the commands, depending on the gaze direction detected, comprising commands for moving such a robot aim in order to vary the viewing frame of the telecamera.

16. The system of claim 15, wherein the telecamera is an endoscopic telecamera.

17. A method for controlling a robotized surgery system comprising robot arms that act under the control of manipulators of a control console comprising a screen and intended for a surgeon, comprising the steps of:

detecting the direction of the surgeon'"'"'s gaze and controlling functions of the surgery system depending on the gaze direction detected;

showing on the screen one or more pictures of the operating field;

detecting the distance of the surgeon'"'"'s eyes from the screen and

varying the enlargement of the pictures on the screen, increasing the picture size on the screen when the distance reduces, varying the position of the picture depending on the distance detected, signalling whether the distance detected is within a predefined working space of the distance detection system, whether it is within a predefined viewing space of the picture of the operating field, or combinations thereof, and wherein assignment of the manipulators and the robot arms is performed by an eye tracking system.

18. The method according to claim 17, comprising the steps of:

showing the pictures of the operating field on at least one zone of the screen;

preventing the movement of the robot arms when the gaze direction detected does not fall within the screen or at least said screen zone and/or enabling the movement of the at least one robot arm when the direction of the gaze detected falls within the screen or at least said screen zone.

19. The method according to claim 17, comprising the steps of:

displaying on the screen selection areas associated with commands for the surgery system and, when a gaze direction which falls within one of said selection areas is detected, selecting the command associated with this area.

20. The method according to claim 17, comprising the steps of:

showing pictures of the robot arms on the screen; and

, when a new robot arm has to be assigned to one manipulator, selecting with one'"'"'s gaze a picture of the new arm shown on the screen and then moving the one'"'"'s gaze direction on the screen into a screen position showing the picture of the robot arm assigned to said one manipulator to command the system to move the selected picture as dragging by means of one'"'"'s gaze movement and assigning the new robot arm to said one manipulator.

21. The method according to claim 17, comprising the steps of:

showing on the screen a picture of the operating field recorded by a telecamera, moved by a robot arm, and moving the robot arm for varying the viewing frame of the telecamera depending on the detected direction of the gaze on the screen picture.

22. The method of claim 18, wherein a further manual command confirming enabling is sent.

23. The method of claim 19, wherein the command selected is also confirmed by means of operation of a further manual confirmation device.

24. A robotized surgery system comprising at least one robot arm which acts under the control of manipulators of a control console which is intended for a surgeon, characterized in that the console comprises an eye tracking system for detecting the direction of the surgeon'"'"'s gaze and for entering commands depending on the directions of the gaze detected, and

further comprising a distance detection system for detecting the distance between the surgeon'"'"'s eyes and a screen showing a picture of the operating field, and

wherein the distance detection system varies the size of the picture on the screen, increases the picture size on the screen when the distance reduces, varies the position of the picture of the operating field shown on the screen depending on a variation in the distance detected, or combinations thereof, and wherein the assignment of the manipulators and the at least one robot arm is performed by the eye tracking system.

25. The system according to claim 24, characterized in that the console comprises a screen with at least one zone for viewing the operating field and in that the tracking system generates a command for disabling the movement of at least one robot arm when a gaze direction which falls outside the screen or at least said zone of the screen is detected.

26. The system according to claim 24, characterized in that the console comprises a screen with at least one zone for viewing the operating field and a human machine interface which displays on the screen selection areas which are associated with commands, the tracking system estimating the direction of the surgeon'"'"'s gaze and performing the selection of commands associated with one area of the selection areas when it detects a gaze direction which falls within said one area.

27. The system according to claim 26, characterized in that the console comprises manipulators for operating the robot arms and the commands associated with the selection areas comprise commands for mutual assignment of manipulators and robot arms.

28. The system according to claim 25, characterized in that the screen forms part of a system for viewing the operating field in 3D.

29. The system according to claim 24, characterized in that the eye tracking system comprises at least one telecamera for recording the picture of at least the surgeon'"'"'s eyes and means for calculating the direction of the gaze depending on the picture taken.

30. The system according to claim 24, characterized in that the console comprises a screen with at least one zone for viewing the operating field and in that the tracking system generates a command for enabling the movement of at least one robot arm when a gaze direction which falls inside the screen or at least said zone of the screen is detected, this command being combined with a manual command for confirming enabling in order to activate movement of the robot arm.

31. The system according to claim 24, characterized in that the console comprises a screen with at least one zone for viewing the operating field recorded by a telecamera, which is moved by a robot arm, the commands, depending on the gaze direction detected, comprising commands for moving such a robot arm in order to vary the viewing frame of the telecamera.