Medical robot arm apparatus, medical robot arm control system, medical robot arm control method, and program

• US 11,129,682 B2
• Filed: 07/15/2015
• Issued: 09/28/2021
• Est. Priority Date: 09/24/2013
• Status: Active Grant

• Current Assignee: Sony Olympus Medical Solutions Inc. (Sony Group Corp.)
• Original Assignee: Sony Olympus Medical Solutions Inc. (Sony Group Corp.)
• Inventors: Fukushima, Tetsuharu, Kokubo, Wataru, Tsuboi, Toshimitsu, Miyamoto, Atsushi, Nagasaka, Kenichiro, Hirose, Kenji
• Primary Examiner(s): Carey, Michael J
• Assistant Examiner(s): Frankert, Genja M
• Application Number:
  US14/800,418
• Publication Number:
  US 20150313679A1
• Time in Patent Office: 2,267 Days
• CPC Class Codes: A61B 1/00149 using articulated arms
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 2090/066 for measuring torque
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 90/06 Measuring instruments not otherwise provided for
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 9/1633 compliant, force, torque control, e.g. combined with position control
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/39237 Torque disturbance control
• CPC Class Codes: G05B 2219/40599 Force, torque sensor integrated in joint
• CPC Class Codes: G05B 2219/40613 Camera, laser scanner on end effector, hand eye manipulator, local
• CPC Class Codes: G05B 2219/41397 Estimate voltage control signal as function of voltage control signal and position error

Abstract

Provided is a surgical imaging apparatus that includes a multi-link, multi joint structure including a plurality of joints that interconnect a plurality of links to provide the multi-link, multi joint structure with a plurality of degrees of freedom, at least one video camera being disposed on a distal end of the multi-link, multi-joint structure; at least one actuator that drives at least one of the plurality of joints; and circuitry that detects a joint force experienced at the at least one of the plurality of joints in response to an applied external force, and controls the at least one actuator based on the joint force so as to position the video camera.

Claims

1. A surgical imaging system comprising:

a multi-link, multi-joint structure including a plurality of joints that interconnect a plurality of links to provide the multi-link, multi-joint structure with a plurality of degrees of freedom, at least one video camera being disposed on a distal end of the multi-link, multi-joint structure, the at least one video camera configured to capture a three-dimensional (3D), optically magnified image having a resolution of at least 3840×

2160 pixels (4K resolution);

at least one actuator that drives at least one of the plurality of joints;

at least one display having a resolution of at least 3840×

2160 pixels (4K resolution) that displays the 3D, optically magnified, 4K resolution image captured by the video camera; and

circuitry configured to;

control the at least one actuator such that a motion of the multi-link, multi-joint structure enables the video camera to maintain aim at a point, which is away from the video camera by a focal length, during movement of the video camera, and

control the video camera to autofocus the 3D, optically magnified, 4K resolution image captured by the video camera aimed at the point.

2. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the at least one actuator to move the video camera in a direction of an applied external force.

3. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the at least one actuator to move the video camera such that a field of view of the video camera is adjusted.

4. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control a movement speed of the video camera based on a magnitude of an applied external force.

5. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the video camera to autofocus during a movement of the video camera.

6. The surgical imaging system according to claim 1, further comprising at least one sensor that detects a joint force, the sensor being disposed at the at least one of the plurality of joints,

wherein the circuitry is further configured to detect the joint force based on a signal output by the at least one sensor.

7. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to digitally magnify and reduce the 3D image.

8. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the video camera to optically magnify and reduce the 3D image.

9. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the at least one actuator such that a position of the video camera is stabilized.

10. The surgical imaging system according to claim 9,

wherein the circuitry is further configured to control the at least one actuator to deactivate the video camera stabilization in response to an input.

11. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the at least one actuator such that a viscous drag coefficient on a rotary motion of the at least one of the plurality of joints in response to an applied external force is adjusted.

12. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the at least one actuator such that a ratio of an applied external force to a travel distance of the at least one of the plurality of joints in response to the applied external force is adjusted.

13. The surgical imaging system according to claim 12,

wherein the circuitry is further configured to store at least two different values for the ratio, and switch the value of the ratio between the at least two different values in response to an input.

14. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the at least one actuator such that the video camera moves only on a plane of a cone having a certain point in a space as an apex, and such that a photographing direction of the video camera is fixed on the certain point, in response to an input.

15. The surgical imaging system according to claim 14,

wherein a distance between the video camera and the certain point is maintained constant.

16. The surgical imaging system according to claim 15,

wherein the certain point is a focus position of the video camera.

17. The surgical imaging system according to claim 1, further comprising a switch disposed at the distal end that receives an input.

18. The surgical imaging system according to claim 1,

wherein display size of the display is greater than or equal to 55 inches.

19. The surgical imaging system according to claim 1,

wherein the circuitry is further configured to control the at least one actuator based on a control value for whole body cooperative control of the multi-link, multi joint structure.

20. The surgical imaging system according to claim 19,

wherein the circuitry is further configured to calculate the control value according to generalized inverse dynamics using a state of the multi-link structure acquired based on a joint force as detected by the circuitry, the purpose of motion of the a multi-link, multi-joint structure, and a constraint condition of the multi-link, multi-joint structure.

21. The surgical imaging system according to claim 20,

wherein the circuitry is further configured to calculate the control value based on a virtual force to achieve the purpose of motion, and an actual force obtained by converting the virtual force into force for driving the at least one of the plurality of joints based on the constraint condition.

22. The surgical imaging system according to claim 20,

wherein the circuitry controls the at least one actuator based on a command value to achieve an ideal response of the multi-link, multi joint structure, the command value being calculated by correcting the control value to compensate for influence of a disturbance on at least one of the plurality of joints.

23. The surgical imaging system according to claim 20,

wherein the circuitry is configured to control the at least one actuator such that gravity acting on the multi-link, multi-joint structure is negated, and that movement of the multi-link, multi-joint structure in a direction of an applied external force is supported.

24. The surgical imaging system according to claim 1,

wherein the circuitry is configured to detect a rotational angle of at least one of the plurality of joints.

25. The surgical imaging system according to claim 1,

wherein the circuitry is configured to detect external torque applied to the at least one of the plurality of joints and generated torque generated by the at least one actuator.

26. The surgical imaging system according to claim 1,

wherein the video camera includes a 4K CMOS sensor.

27. The surgical imaging system according to claim 1,

wherein the video camera is an endoscope.

28. The surgical imaging system according to claim 1,

wherein the multi-link, multi-joint structure has six or more degrees of freedom.

29. The surgical imaging system according to claim 1, wherein the movement of the video camera is movement within a surface of a cone having a same part as an apex, and an axis of the cone is used as a pivot axis in a state in which a photographing direction of the video camera is fixed to the same part.

30. The surgical imaging system according to claim 1, wherein the focus is maintained even when the distance between an object in focus and the video camera is changed.

31. The surgical imaging system according to claim 1, wherein the circuitry is configured to control the at least one actuator such that the video camera continues to photograph a same part even after movement of the video camera with respect to an object in focus such that the object in focus remains within a frame of the at least one display.

32. The surgical imaging system according to claim 1, wherein a locked distance and the point on a same part are set to a focal length and a focal point of the video camera respectively.