ROBOTIZED SURGERY SYSTEM WITH IMPROVED CONTROL

• US 20130030571A1
• Filed: 04/01/2011
• Published: 01/31/2013
• Est. Priority Date: 04/07/2010
• Status: Active Grant

• Current Assignee: Asensus Surgical Italia Srl (Asensus Surgical, Inc.)
• Original Assignee: Sofar S.p.a. (Alfasigma S.p.A.)
• Inventors: Invernizzi, Paolo, Ruiz Morales, Emilio, Brasset, Damien
• Granted Patent: US 9,360,934 B2
• Time in Patent Office: Days
• US Class Current: 700/259
• CPC Class Codes: A61B 17/00 Surgical instruments, devices or methods, e.g. tourniquets A61B18/00 takes precedence; contraceptive devices, pessaries, or applicators therefor A61F6/00; eye surgery A61F9/007; ear surgery A61F11/00
• CPC Class Codes: A61B 2017/00216 with eye tracking or head position tracking control
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 5/11 Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb for measuring pulse A61B5/02 ;A61B5/1038 takes precedence; motion detection to correct for motion artifacts in physiological signals A61B5/721
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A robotized surgery system (10) comprises at least one robot arm (11) which acts under the control of a control console (12) intended for the surgeon. The console (12) comprises an eye tracking system (21) for detecting the direction of the surgeon'"'"'s gaze and for entering commands depending on the directions of the gaze detected. The console (22) comprises advantageously a screen (23) with at least one zone (23) for viewing the operating field and, among the commands which can be performed depending on the gaze directions, there is advantageously an automatic command for enabling or disabling the movement of the robot arm (11) when a gaze direction which falls within or outside of said zone (23) of the screen is detected.

Claims

1-15. -15. (canceled)

16. Robotized surgery system (10) comprising at least one robot arm (11) which acts under the control of a control console (12) which is intended for the surgeon, characterized in that the console (12) comprises an eye tracking system (21) for detecting the direction of the surgeon'"'"'s gaze and for entering commands depending on the directions of the gaze detected, the console (12) comprises a screen (22) with at least one zone for viewing the operating field and a human machine interface which displays on the screen selection areas (31) which are associated with commands, the tracking system estimating the direction of the surgeon'"'"'s gaze and performing the selection of commands associated with one area of the selection areas (31) when it detects a gaze direction which falls within said one area and the console comprises manipulators (17, 18) for operating the robot arms and the commands associated with the selection areas (31) comprise commands for mutual assignment of manipulators and robot arms.

17. System according to claim 16, characterized in that the console (12) comprises a screen (22) with at least one zone (23) for viewing the operating field and in that the tracking system (21) generates a command for disabling the movement of at least one robot arm (11) when a gaze direction which falls outside the screen or at least said zone (23) of the screen is detected.

18. System according to claim 16, characterized in that the screen (22) forms part of a system for viewing the operating field in 3D.

19. System according to claim 16, characterized in that the eye tracking system comprises at least one telecamera (32, 33) for recording the picture of at least the surgeon'"'"'s eyes and means (21) for calculating the direction of the gaze depending on the picture taken.

20. System according to claim 16, characterized in that it comprises a system (21) for detecting the distance between the surgeon'"'"'s eyes and a screen (22) showing a picture of the operating field, this distance detection system performing a variation in the enlargement and/or position of the picture of the operating field shown on the screen (22) depending on a variation in the distance detected.

21. System according to claim 16, characterized in that the console (12) comprises a screen (22) with at least one zone (23) for viewing the operating field and in that the tracking system (21) generates a command for enabling the movement of at least one robot arm (11) when a gaze direction which falls inside the screen or at least said zone (23) of the screen is detected, this command being combined with a manual command for confirming enabling in order to activate movement of the robot arm.

22. System according to claim 16, characterized in that the console (12) comprises a screen (22) with at least one zone (23) for viewing the operating field recorded by a telecamera (14), preferably an endoscopic telecamera, which is moved by a robot arm (11), the commands, depending on the gaze direction detected, comprising commands for moving such a robot arm in order to vary the viewing frame of the telecamera.

23. Method for controlling a robotized surgery system comprising at least one robot arm (11) which acts under the control of a control console (12) intended for the surgeon, wherein the direction of the surgeon'"'"'s gaze is detected and functions of the surgery system are controlled depending on the gaze direction detected and wherein a new robot arm chosen from among several robot arms is assigned to a control manipulator by selecting with one'"'"'s gaze a picture of the new arm shown on a screen and dragging by means of one'"'"'s gaze this picture into a screen position showing the picture of the robot assigned to this manipulator.

24. Method according to claim 23, wherein pictures of the operating field are shown on at least one zone of a screen and the movement of the at least one robot arm is prevented when the gaze direction detected does not fall within the screen or at least said screen zone and/or the movement of the at least one robot arm is enabled when the direction of the gaze detected falls within the screen or at least said screen zone and, preferably, a further manual command confirming enabling is also sent.

25. Method according to claim 23, wherein selection areas associated with commands for the surgery system are displayed on a screen and, when a gaze direction which falls within one of said selection areas is detected, the command associated with this area is selected and, preferably, the command selected is also confirmed by means of operation of a further manual confirmation device.

26. Method according to claim 23, wherein a picture of the operating field is shown on a screen, the distance of the surgeon'"'"'s eyes from the screen is detected and the enlargement and/or position of the picture is varied depending on the distance detected and/or it is signalled whether the distance detected is within a predefined working space of the distance detection system and/or whether it is within a predefined viewing space of the picture of the operating field, preferably in 3D.

27. Method according to claim 23, wherein a picture of the operating field recorded by a telecamera, preferably an endoscopic telecamera, moved by a robot arm (11) is shown on a screen, and the robot arm varies the viewing frame of the telecamera depending on the detected direction of the gaze on the screen picture.

28. Robotized surgery system (10) comprising at least one robot arm (11) which acts under the control of a control console (12) which is intended for the surgeon, characterized in that the console (12) comprises an eye tracking system (21) for detecting the direction of the surgeon'"'"'s gaze and for entering commands depending on the directions of the gaze detected and in that it comprises a system (21) for detecting the distance between the surgeon'"'"'s eyes and a screen (22) showing a picture of the operating field, this distance detection system performing a variation in the enlargement and/or position of the picture of the operating field shown on the screen (22) depending on a variation in the distance detected.

29. System according to claim 28, characterized in that the console (12) comprises a screen (22) with at least one zone (23) for viewing the operating field and in that the tracking system (21) generates a command for disabling the movement of at least one robot arm (11) when a gaze direction which falls outside the screen or at least said zone (23) of the screen is detected.

30. System according to claim 28, characterized in that the console (12) comprises a screen (22) with at least one zone for viewing the operating field and a human machine interface which displays on the screen selection areas (31) which are associated with commands, the tracking system estimating the direction of the surgeon'"'"'s gaze and performing the selection of commands associated with one area of the selection areas (31) when it detects a gaze direction which falls within said one area.

31. System according to claim 30, characterized in that the console comprises manipulators (17, 18) for operating the robot arms and the commands associated with the selection areas (31) comprise commands for mutual assignment of manipulators and robot arms.

32. System according to claim 29, characterized in that the screen (22) forms part of a system for viewing the operating field in 3D.

33. System according to claim 28, characterized in that the eye tracking system comprises at least one telecamera (32, 33) for recording the picture of at least the surgeon'"'"'s eyes and means (21) for calculating the direction of the gaze depending on the picture taken.

34. System according to claim 28, characterized in that the console (12) comprises a screen (22) with at least one zone (23) for viewing the operating field and in that the tracking system (21) generates a command for enabling the movement of at least one robot arm (11) when a gaze direction which falls inside the screen or at least said zone (23) of the screen is detected, this command being combined with a manual command for confirming enabling in order to activate movement of the robot arm.

35. System according to claim 28, characterized in that the console (12) comprises a screen (22) with at least one zone (23) for viewing the operating field recorded by a telecamera (14), preferably an endoscopic telecamera, which is moved by a robot arm (11), the commands, depending on the gaze direction detected, comprising commands for moving such a robot arm in order to vary the viewing frame of the telecamera.

36. Method for controlling a robotized surgery system comprising at least one robot arm (11) which acts under the control of a control console (12) intended for the surgeon, wherein the direction of the surgeon'"'"'s gaze is detected and functions of the surgery system are controlled depending on the gaze direction detected and wherein a picture of the operating field is shown on a screen, the distance of the surgeon'"'"'s eyes from the screen is detected and the enlargement and/or position of the picture is varied depending on the distance detected and/or it is signalled whether the distance detected is within a predefined working space of the distance detection system and/or whether it is within a predefined viewing space of the picture of the operating field, preferably in 3D.

37. Method according to claim 36, wherein pictures of the operating field are shown on at least one zone of a screen and the movement of the at least one robot arm is prevented when the gaze direction detected does not fall within the screen or at least said screen zone and/or the movement of the at least one robot arm is enabled when the direction of the gaze detected falls within the screen or at least said screen zone and, preferably, a further manual command confirming enabling is also sent.

38. Method according to claim 36, wherein selection areas associated with commands for the surgery system are displayed on a screen and, when a gaze direction which falls within one of said selection areas is detected, the command associated with this area is selected and, preferably, the command selected is also confirmed by means of operation of a further manual confirmation device.

39. Method according to claim 36, wherein a new robot arm chosen from among several robot arms is assigned to a control manipulator by selecting with one'"'"'s gaze a picture of the new arm shown on a screen and dragging by means of one'"'"'s gaze this picture into a screen position showing the picture of the robot assigned to this manipulator.

40. Method according to claim 36, wherein a picture of the operating field recorded by a telecamera, preferably an endoscopic telecamera, moved by a robot arm (11) is shown on a screen, and the robot arm varies the viewing frame of the telecamera depending on the detected direction of the gaze on the screen picture.