Manipulator system

• US 9,204,934 B2
• Filed: 01/06/2014
• Issued: 12/08/2015
• Est. Priority Date: 07/15/2011
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Ogawa, Ryohei, Kishi, Kosuke
• Primary Examiner(s): Sample, Jonathan L
• Application Number:
  US14/147,762
• Publication Number:
  US 20140121834A1
• Time in Patent Office: 701 Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2090/368 changing the image on a display according to the operator's position
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/502 Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: B25J 3/00 Manipulators of master-slave type, i.e. both controlling unit and controlled unit perform corresponding spatial movements
• CPC Class Codes: B25J 3/04 involving servo mechanisms servo-actuated heads B25J15/02
• CPC Class Codes: B25J 9/1689 Teleoperation

Abstract

A manipulator system includes a master manipulator configured to send an input command, a slave manipulator configured to operate according to the input command, an image capturing unit configured to acquire an image of an object, a display device placed in front of the operator and configured to display the image acquired by the image capturing unit, a detection device configured to detect the direction of an operator'"'"'s face of the operator with respect to the display device, and a control unit configured to determine whether the direction of the operator'"'"'s face is within a predetermined angle with respect to the display device based on the detection result in the detection device, and to shift an operation mode of the slave manipulator between a first control mode and a second control mode in which an operation is limited more than in the first control mode based on a determination result.

Claims

1. A manipulator system comprising:

a master manipulator configured to send an input command based on manipulation by an operator;

a slave manipulator configured to operate in accordance with the input command;

an image capturing unit configured to acquire an image of an object;

a display device placed in front of the operator and configured to display the image acquired by the image capturing unit;

a detection device configured to detect a direction of an operator'"'"'s face with respect to the display device; and

a control unit configured to determine whether the direction of the operator'"'"'s face is within a predetermined angle with respect to the display device based on a detection result in the detection device, and to shift an operation mode of the slave manipulator between a predetermined first control mode and a second control mode in which an operation is limited more than in the first control mode based on a determination result;

wherein the detection device comprises;

an index unit that is configured to be mountable on the operator to be moved along with the operator'"'"'s face; and

a detection unit positioned with respect to the display device and configured to detect the index unit within a predetermined range, and

wherein the control unit operates the slave manipulator in the second control mode when the index unit is not detected within the predetermined range in the detection unit, the control unit shifts to an input standby mode of an approval signal by the operator to perform a shift from the second control mode to the first control mode when a state in which the index unit is not detected within the predetermined range is changed into a state in which the index unit is detected within the predetermined range, and the control unit shifts from the second control mode to the first control mode when the approval signal is input.

2. The manipulator system according to claim 1, wherein the second control mode is a mode in which the slave manipulator is continuously stopped.

3. The manipulator system according to claim 1, wherein

the detection device detects a variation of the direction of the operator'"'"'s face in a horizontal direction in a state in which the display device is placed in front of the operator;

and

the control unit shifts to the second control mode when the detection device detects that the direction of the operator'"'"'s face is varied more than the predetermined angle to the horizontal direction.

4. The manipulator system according to claim 1, wherein the control unit sends a message to prompt the operator to move the operator'"'"'s face or the display device such that the direction of the operator'"'"'s face is within a range of the predetermined angle with respect to the display device by using device which is capable of being recognized by the operator, when the control unit determines that the direction of the operator'"'"'s face deviates from the range of the predetermined angle with respect to the display device.

5. The manipulator system according to claim 1, wherein the control unit sends a message to show a moving direction of the operator'"'"'s face or a moving direction of the display device such that the direction of the operator'"'"'s face is within a range of the predetermined angle with respect to the display device by using device which is capable of being recognized by the operator, when the control unit determines that the direction of the operator'"'"'s face deviates from the range of the predetermined angle with respect to the display device.

6. The manipulator system according to claim 4, wherein the control unit displays the message on the display device.

7. The manipulator system according to claim 1, wherein

the display device is provided with a movable mechanism configured to movably support the display device, and

the control unit moves the display device by using the movable mechanism such that the direction of the operator'"'"'s face is within a range of the predetermined angle with respect to the display device, when the control unit determines that the direction of the operator'"'"'s face deviates from the range of the predetermined angle with respect to the display device.

8. The manipulator system according to claim 1, wherein the second control mode is a mode in which the slave manipulator is operated at an operation speed lower than the first control mode.

9. The manipulator system according to claim 1, wherein

the display device displays two images on the same display surface based on binocular disparity, and

a portion of the detection device is installed at 3D glasses configured to be mountable on the operator and configured to form a stereoscopic image based on the two images.

10. The manipulator system according to claim 9, wherein

the detection device detects a distance and a direction of the 3D glasses with respect to the display device, and

the control unit has initial position information showing a range of the distance and direction in which the stereoscopic image is capable of being optimally observed among the distance and the direction, the control unit shifts to the first control mode based on the initial position information, when the distance and the direction of the 3D glasses detected by the detection device are within the range.

11. The manipulator system according to claim 1, wherein the control unit performs a shift from the second control mode to the first control mode without input of the approval signal when the direction of the operator'"'"'s face is within the range of the predetermined angle with respect to the display device within a predetermined time from when the first control mode is shifted to the second control mode.

12. The manipulator system according to claim 1, wherein the control unit performs a shift from the second control mode to the first control mode without input of the approval signal when the direction of the operator'"'"'s face is within the range of the predetermined angle with respect to the display device while an operation amount of the master manipulator from when the shift from the first control mode to the second control mode is performed is within a predetermined operation amount.

13. The manipulator system according to claim 1, wherein

the index unit has a predetermined color or a predetermined shape,

the detection unit includes an imaging unit configured to acquire an image including the index unit, and

the control unit sets the predetermined range on the image and recognizes an area having the predetermined color or the predetermined shape in the image as the index unit.

14. The manipulator system according to claim 1, wherein

the index unit includes a plurality of index elements spaced apart from each other in at least one of a horizontal direction or a vertical direction of the operator wearing the index unit, and

the detection unit detects the direction of the index unit based on a result in which positions of the plurality of index elements are detected.