Virtual reality surgical device

• US 10,285,765 B2
• Filed: 05/05/2015
• Issued: 05/14/2019
• Est. Priority Date: 05/05/2014
• Status: Active Grant

• Current Assignee: Vicarious Surgical, Inc.
• Original Assignee: Vicarious Surgical, Inc.
• Inventors: Sachs, Adam, Khalifa, Sammy, Greene, Barry Stuart
• Primary Examiner(s): Holwerda, Stephen
• Application Number:
  US15/305,035
• Publication Number:
  US 20170181802A1
• Time in Patent Office: 1,470 Days
• Field of Search: 7449001, 7449002, 7449003, 7449004, 7449005, 7449006, 7449007, 600141, 600142, 600146, 604528
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 17/29 Forceps for use in minimally invasive surgery
• CPC Class Codes: A61B 17/2909 Handles
• CPC Class Codes: A61B 17/3417 Details of tips or shafts, e.g. grooves, expandable, bendable; Multiple coaxial sliding cannulas, e.g. for dilating syringe needles A61M5/32; dilators A61M29/00
• CPC Class Codes: A61B 17/3462 with means for changing the diameter or the orientation of the entrance port of the cannula, e.g. for use with different-sized instruments, reduction ports, adapter seals expandable cannulas A61B17/3417
• CPC Class Codes: A61B 2017/00207 with hand gesture control or hand gesture recognition
• CPC Class Codes: A61B 2017/00216 with eye tracking or head position tracking control
• CPC Class Codes: A61B 2017/00283 with a device releasably connected to an inner wall of the abdomen during surgery, e.g. an illumination source
• CPC Class Codes: A61B 2017/00398 using powered actuators, e.g. stepper motors, solenoids
• CPC Class Codes: A61B 2017/2919 details of linkages or pivot points
• CPC Class Codes: A61B 2017/3445 used as instrument channel for multiple instruments catheters A61B2017/22072
• CPC Class Codes: A61B 2034/301 for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
• CPC Class Codes: A61B 2034/302 specifically adapted for manipulations within body cavities, e.g. within abdominal or thoracic cavities
• CPC Class Codes: A61B 2034/306 Wrists with multiple vertebrae
• CPC Class Codes: A61B 2034/715 Cable tensioning mechanisms for removing slack
• CPC Class Codes: A61B 2034/741 Glove like input devices, e.g. "data gloves"
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 2090/306 using optical fibres
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 2090/368 : changing the image on a display according to the operator's position
• CPC Class Codes: A61B 2090/371 : with simultaneous use of two cameras
• CPC Class Codes: A61B 2090/372 : Details of monitor hardware
• CPC Class Codes: A61B 2090/502 : Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 34/20 : Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 : Surgical robots
• CPC Class Codes: A61B 34/35 : for telesurgery
• CPC Class Codes: A61B 34/37 : Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/71 : Manipulators operated by drive cable mechanisms
• CPC Class Codes: A61B 34/74 : Manipulators with manual electric input means
• CPC Class Codes: A61B 34/76 : Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 90/06 : Measuring instruments not otherwise provided for
• CPC Class Codes: A61B 90/30 : Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
• CPC Class Codes: A61B 90/361 : Image-producing devices, e.g. surgical cameras
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation
• CPC Class Codes: B25J 9/104 : with cables, chains or ribbons
• CPC Class Codes: B25J 9/108 : Bearings specially adapted therefor bearings in general F16C
• CPC Class Codes: G02B 27/017 : Head mounted
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

A system for use in surgery includes a central body, a visualization system operably connected to the central body, a video rendering system, a head-mounted display for displaying images from the video rendering system, a sensor system, and a robotic device operably connected to the central body. The visualization system includes at least one camera and a pan system and/or a tilt system. The sensor system tracks the position and/or orientation in space of the head-mounted display relative to a reference point. The pan system and/or the tilt system are configured to adjust the field of view of the camera in response to information from the sensor system about changes in at least one of position and orientation in space of the head-mounted display relative to the reference point.

Claims

1. A system for use in surgery comprising:

a central body;

a visualization system operably connected to the central body comprising;

at least a first camera, and

at least one of a pan system or a tilt system;

a first processor and machine readable memory comprising instructions that when executed cause the first processor to generate images based on information from the at least one camera;

a head-mounted display for displaying images generated by the first processor;

a first sensor for tracking at least one of the position in space of the head-mounted display relative to a first reference point or the orientation in space of the head-mounted display relative to the first reference point, wherein at least one of the pan system or the tilt system are configured to adjust the field of view of the camera in response to information from the first sensor about changes in at least one of position or orientation in space of the head-mounted display relative to the first reference point;

a robotic device operably connected to the central body, wherein the robotic device comprises;

a first rotational actuator for rotating a first portion of the robotic device with respect to a second portion of the robotic device;

a first hinged actuator for changing the angle between a third portion of the robotic device and a fourth portion of the robotic device operably coupled to the first rotational actuator;

a second rotational actuator for rotating a fifth portion of the robotic device with respect to a sixth portion of the robotic device operably coupled to the first hinged actuator;

a second hinged actuator for changing the angle between a seventh portion of the robotic device and an eighth portion of the robotic device operably coupled to the second rotational actuator;

a third rotational actuator for rotating a ninth portion of the robotic device with respect to a tenth portion of the robotic device operably coupled to the second hinged actuator;

a third hinged actuator for changing the angle between an eleventh portion of the robotic device and a twelfth portion of the robotic device operably coupled to the third rotational actuator; and

a surgical end-effector operably coupled to the third hinged actuator;

a second sensor for tracking a series of changes in a position of a portion of an arm of a user;

a second processor and machine readable memory comprising instructions that when executed cause the second processor to determine a series of positions and orientations of at least two of the twelve portions of the robotic device in response to the series of changes in the position of the portion of the arm of the user based on the degrees of freedom of a human arm, whereby the determined series of positions and orientations of the portions of the robotic device replicate a human-like motion achievable by the human arm; and

at least one servomotor for adjusting at least one of positions or orientations of at least one of the first rotational actuator, the second rotational actuator, the third rotational actuator, the first hinged actuator, the second hinged actuator, the third hinged actuator, or the surgical end-effector to cause the robotic device to follow the series of changes in position of the portion of the arm of the user according to the determined series of positions and orientations of the at least two of the twelve portions of the robotic device so that at least one of the twelve portions of the robotic device mimics the motion of the arm of the user.

2. The device of claim 1, the first processor and machine readable memory comprising instructions that when executed cause the first processor to digitally adjust the field of view of the generated images based on information from the first sensor.

3. The device of claim 1, the visualization system further comprising a second camera.

4. The device of claim 3, the images generated by the first processor comprising stereoscopic images based on information from the first and second cameras.

5. The device of claim 3, wherein, in an insertion configuration, cross-sectional dimensions of the visualization system in a plane normal to an insertion axis are smaller than a center distance between the first camera and the second camera along the insertion axis.

6. The device of claim 1, comprising a plurality of cameras.

7. The device of claim 6, the first processor and machine readable memory comprising instructions that when executed cause the first processor to generate the images based on software interlacing of signal information from the plurality of cameras.

8. The device of claim 1, further comprising at least one camera sensor to measure at least one of the position and orientation of the camera.

9. The device of claim 1, wherein the robotic device further comprises a positional actuator for changing the position of the robotic device relative to the central body such that the robotic device may be used on either a first side or a second side of the central body.

10. The device of claim 1, further comprising a second robotic device comprising:

a fourth rotational actuator for rotating a first portion of the second robotic device with respect to a second portion of the second robotic device;

a fourth hinged actuator for changing the angle between a third portion of the second robotic device and a fourth portion of the second robotic device operably coupled to the fourth rotational actuator;

a fifth rotational actuator for rotating a fifth portion of the second robotic device with respect to a sixth portion of the second robotic device operably coupled to the fourth hinged actuator;

a fifth hinged actuator for changing the angle between a seventh portion of the second robotic device and an eighth portion of the second robotic device operably coupled to the fifth rotational actuator;

a sixth rotational actuator for rotating a ninth portion of the second robotic device with respect to a tenth portion of the second robotic device operably coupled to the fifth hinged actuator;

a sixth hinged actuator for changing the angle between an eleventh portion of the second robotic device and a twelfth portion of the second robotic device operably coupled to the sixth rotational actuator; and

a second surgical end-effector operably coupled to the sixth hinged actuator.

11. The system of claim 1, wherein at least one of the first hinged, second hinged, and third hinged robotic actuators comprises:

a first body comprising;

a proximal connection component coupling the robotic actuator to proximal systems, and

a first bearing surface;

a second body comprising;

a distal connection component coupling the robotic actuator to distal systems, and

a second bearing surface forming a bearing with the first bearing surface whereby the bearing constrains the motion of the first body relative to the motion of the second body in at least one degree of freedom;

at least one of a pulley and a capstan operably coupled with at least one of the first body and the second body;

an actuator cable configured to actuate at least one of the pulley and the capstan; and

at least one contoured surface defined by the robotic actuator and forming a contoured pathway to allow a plurality of additional cables to pass through the pathway from systems coupled to the proximal connection component to systems coupled to the distal connection component wherein a shape and a position of the pathway is such that lengths of the additional cables remain substantially constant for substantially an entire range of motion for which the robotic actuator is used.

12. The system of claim 1, wherein at least one of the first hinged, second hinged, and third hinged robotic actuators comprises:

a first body comprising a proximal connection component;

a second body comprising a distal connection component;

a bearing system constraining the motion of the first body relative to the second body in all degrees of freedom except rotation about one axis perpendicular to the distal-proximal axis of the robotic actuator;

at least one of a pulley or a capstan operably coupled with at least one of the first body and the second body;

an actuator cable configured to actuate the at least one of the pulley and the capstan; and

at least one contoured surface defined by the robotic actuator and forming a contoured pathway to allow a plurality of additional cables to pass through the pathway from systems coupled to the proximal connection component to systems coupled to the distal connection component wherein a shape and a position of the pathway is such that lengths of the additional cables remain substantially constant for substantially an entire range of motion for which the robotic actuator is used.

13. The system of claim 1, wherein at least one of the first rotational, second rotational, and third rotational robotic actuators comprises:

a first body comprising a proximal connection component;

a second body comprising a distal connection component;

a bearing system constraining the motion of the first body relative to the second body in all degrees of freedom except rotation about the distal-proximal axis of the robotic actuator;

at least one of a pulley or a capstan operably coupled with at least one of the first body or the second body;

an actuator cable configured to actuate the at least one of the pulley and the capstan; and

a hole defined by the robotic actuator with an inner diameter of at least three times the diameter of the actuator cable configured such that additional cables may pass through the hole from systems coupled to the proximal connection component to systems coupled to the distal connection component wherein a shape and a position of the hole is such that lengths of the additional cables remain substantially constant for substantially an entire range of motion for which the robotic actuator is used.

14. The system of claim 1, wherein the surgical end-effector comprises:

a main grasper body;

a first grasper jaw operably coupled to the main grasper body;

a second grasper jaw operably coupled to the main grasper body;

an actuation cable; and

a linkage mechanism coupling at least one of the first grasper jaw and the second grasper jaw with the actuation cable wherein the linkage mechanism provides for non-linear movement of a distal end of at least one of the first grasper jaw or the second grasper jaw in response to movement of the actuation cable.

15. The surgical grasper of claim 14, further comprising a strain gauge fixed to at least one of the main grasper body, the first grasper jaw, the second grasper jaw, the actuation cable, and the linkage mechanism wherein a force value between the distal end of the first grasper jaw and the distal end of the second grasper jaw is determined based on information from the strain gauge.

16. The surgical grasper of claim 15, further comprising an operator interface including a haptic feedback device for providing haptic feedback to a user of the operator interface based on the information from the strain gauge.

17. The surgical grasper of claim 14, further comprising at least one of a spring operably coupled with at least one of the first grasper jaw and the second grasper jaw.

18. The surgical grasper of claim 14, further comprising at least one of software and hardware control loops for controlling at least one of the force of the grasper jaws and the position of the grasper jaws.

19. The surgical grasper of claim 14, further comprising at least one of a servomotor operably coupled with the actuation cable.

20. The surgical grasper of claim 14, further comprising at least one of a position sensor for measuring the position of at least one of the first grasper jaw and the second grasper jaw.