Method and apparatus for transforming coordinate systems in a telemanipulation system

• US 7,333,642 B2
• Filed: 04/15/2003
• Issued: 02/19/2008
• Est. Priority Date: 01/21/1992
• Status: Expired due to Fees

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Green, Philip S.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): CONOVER, DAMON M
• Application Number:
  US10/414,814
• Publication Number:
  US 20030228039A1
• Time in Patent Office: 1,771 Days
• Field of Search: 382/128, D24/138, 600/118
• US Class Current: 382/128
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/35 for telesurgery
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/70 Manipulators specially adapted for use in surgery
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 3/04 involving servo mechanisms servo-actuated heads B25J15/02
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G03C 1/0053 with high content of silver chloride
• CPC Class Codes: G05B 2219/35506 Camera images overlayed with graphics, model
• CPC Class Codes: G05B 2219/39389 Laparoscopic surgery, camera on center of operated part, view around, scale
• CPC Class Codes: G05B 2219/40158 Correlate actual image at angle with image presented to operator without angle
• CPC Class Codes: G05B 2219/40161 Visual display of machining, operation, remote viewing
• CPC Class Codes: G05B 2219/40169 Display of actual situation at the remote site
• CPC Class Codes: G05B 2219/45123 Electrogoniometer, neuronavigator, medical robot used by surgeon to operate
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 : Transmission of image signals
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 : Calibration of cameras
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/337 : using polarisation multiplexing
• CPC Class Codes: H04N 13/341 : using temporal multiplexing
• CPC Class Codes: H04N 13/398 : Synchronisation thereof; Control thereof

Abstract

In a telemanipulation system for manipulating objects located in a workspace at a remote worksite by an operator from an operator'"'"'s station, such as in a remote surgical system, the remote worksite having a manipulator with an end effector for manipulating an object at the workspace, such as a body cavity, a controller including a hand control at the control operator'"'"'s station for remote control of the manipulator, an image capture device, such as a camera, and image output device for reproducing a viewable real-time image, the improvement wherein a position sensor associated with the image capture device senses position relative to the end effector and a processor transforms the viewable real-time image into a perspective image with correlated manipulation of the end effector by the hand controller such that the operator can manipulate the end effector and the manipulator as if viewing the workspace in true presence. Image transformation according to the invention includes translation, rotation and perspective correction.

Claims

1. A method comprising:

positioning a surgical end effector within an internal surgical site in a patient body, the surgical end effector operatively associated with a hand controller via a processor so that the processor effects movement of the end effector in response to movement of the hand controller;

capturing an image of the surgical end effector within the internal surgical site with an endoscope;

displaying the image of the end effector so that the end effector defines a displayed position relative to the hand controller;

moving the endoscope within the internal surgical site;

statically realigning the displayed position of the image of the end effector with the hand controller, allowing the operator to view the end effectors as if emerging from the hand controller.

2. The telesurgical method of claim 1, wherein the statically realigning step is performed after the endoscope moving step.

3. The telesurgical method of claim 1, wherein the statically realigning step is performed so as to compensate for the endoscope moving step.

4. The telesurgical method of claim 1, further comprising dynamically realigning movement of the image of the end effector to correspond to movement of the hand controller by altering a coordinate transformation of the processor.

5. A surgical robotic system comprising:

a slave manipulator movable with respect to a surgical site in an actual workspace;

a camera system for capturing an image of the slave manipulator and the surgical site;

a display for displaying a processed image to an operator in an apparent workspace;

a master controller movable with respect to the displayed processed image in the apparent workspace;

a processor for performing a transformation on the captured image to generate a processed image;

wherein;

the orientation of the slave manipulator relative to the surgical site in the displayed processed image defines a first vector;

the orientation of the master controller relative to the displayed processed image of the surgical site in the apparent workspace defines a second vector; and

the transformation on the captured image to generate a processed image aligns the first vector with the second vector.

6. The system of claim 5, the transformation comprising rotation of the captured image about a point on the captured image corresponding to the surgical site.

7. The system of claim 6, the transformation further comprising dynamic realignment of the image such that an apparent origin of the slavemanipulator in the displayed processed image is aligned with the master controller in the apparent workspace.

8. The system of claim 5, the transformation comprising sending of a signal from the processor to the camera instructing the camera to rotate the camera lens about its visual axis.

9. The system of claim 5, the slave manipulator comprising a pair of manipulators, and the master controller comprising a pair of handles.

10. The system of claim 5, the slave manipulator comprising a single end effector, and the master controller comprising a single hand controller.

11. The system of claim 5, the camera system comprising an endoscopic camera.

12. The system of claim 5, the camera system comprising two camera lenses, and the display comprising a stereo video display monitor.

13. The system of claim 12, the two camera lenses separated by 10 degrees.