Method and apparatus for transforming coordinate systems in a telemanipulation system

• US 8,068,649 B2
• Filed: 01/10/2008
• Issued: 11/29/2011
• Est. Priority Date: 01/21/1992
• Status: Expired due to Fees

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Green, Philip S.
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US11/971,984
• Publication Number:
  US 20090082905A1
• Time in Patent Office: 1,419 Days
• Field of Search: 382128-134, 600/118, 600/407, 600/410, 600/101, 600/104, 600/105, 600/117, 600/120, 600/126, 600/132, 600/134, 128/920, 128/922, 348/45, 348/65, 348/72, 700/259, 901/2, 901/47
• US Class Current: 382/128
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/35 for telesurgery
• CPC Class Codes: A61B 34/37 Master-slave robots A61B34/35 takes precedence
• CPC Class Codes: A61B 34/70 Manipulators specially adapted for use in surgery
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 3/04 involving servo mechanisms servo-actuated heads B25J15/02
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G03C 1/0053 with high content of silver chloride
• CPC Class Codes: G05B 2219/35506 Camera images overlayed with graphics, model
• CPC Class Codes: G05B 2219/39389 Laparoscopic surgery, camera on center of operated part, view around, scale
• CPC Class Codes: G05B 2219/40158 Correlate actual image at angle with image presented to operator without angle
• CPC Class Codes: G05B 2219/40161 Visual display of machining, operation, remote viewing
• CPC Class Codes: G05B 2219/40169 Display of actual situation at the remote site
• CPC Class Codes: G05B 2219/45123 Electrogoniometer, neuronavigator, medical robot used by surgeon to operate
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 : Transmission of image signals
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 : Calibration of cameras
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/337 : using polarisation multiplexing
• CPC Class Codes: H04N 13/341 : using temporal multiplexing
• CPC Class Codes: H04N 13/398 : Synchronisation thereof; Control thereof

Abstract

In a telemanipulation system for manipulating objects located in a workspace at a remote worksite by an operator from an operator'"'"'s station, such as in a remote surgical system, the remote worksite having a manipulator with an end effector for manipulating an object at the workspace, such as a body cavity, a controller including a hand control at the control operator'"'"'s station for remote control of the manipulator, an image capture device, such as a camera, and image output device for reproducing a viewable real-time image, the improvement wherein a position sensor associated with the image capture device senses position relative to the end effector and a processor transforms the viewable real-time image into a perspective image with correlated manipulation of the end effector by the hand controller such that the operator can manipulate the end effector and the manipulator as if viewing the workspace in true presence. Image transformation according to the invention includes translation, rotation and perspective correction.

Claims

1. A method for operating a telerobotic system, the system comprising at least one slave manipulator and at least one master manipulator, the slave manipulator having a distal portion and a proximal portion, the system further comprising a display and an endoscope for capturing an image of the slave manipulator, the method comprising:

moving the slave manipulator in response to movement of the master manipulator;

displaying the image of the slave manipulator on the display to an operator controlling the master manipulator; and

transforming the image on the display such that a first vector aligns with a second vector, the first vector defined as originating from the master manipulator pointing toward the distal portion of the slave manipulator in the image, the second vector defined as originating from the proximal portion of the slave manipulator in the image pointing toward the distal portion of the slave manipulator in the image.

2. The method of claim 1, the transforming the image comprising rotating the image.

3. The method of claim 1, the system further comprising a second endoscope for capturing a stereo image of the at least one slave manipulator, the transforming the image on the display comprising transforming a stereo image of the display.

4. The method of claim 1, the distal portion of the slave manipulator coupled to a surgical end effector, the method further comprising:

positioning the surgical end effector within an internal surgical site of a patient; and

positioning the endoscope within the internal surgical site.

5. The method of claim 1, the system further comprising a second slave manipulator and a second master manipulator, the second slave manipulator having a distal portion and a proximal portion, the method further comprising:

moving the second slave manipulator in response to movement of the second master manipulator;

displaying the image of the second slave manipulator on the display to the operator.

6. The method of claim 1, the transforming comprising effecting dynamic synchronization of the position of the distal portion of the slave manipulator in the image with the position of the master manipulator.

7. The method of claim 1, the transforming comprising bringing the position of the slave manipulator in the image in registration with the position of the master manipulator.

8. The method of claim 1, the transforming comprising rotating the endoscope about its visual axis.

9. A telerobotic system comprising:

at least one slave manipulator, the slave manipulator having a distal portion and a proximal portion;

at least one master manipulator, the slave manipulator moved in response to movement of the master manipulator;

an endoscope for capturing an image of the slave manipulator;

a display for displaying the image of the slave manipulator to an operator controlling the master manipulator; and

means for transforming the image on the display such that a first vector aligns with a second vector, the first vector defined as originating from the master manipulator pointing toward the distal portion of the slave manipulator in the image, the second vector defined as originating from the proximal portion of the slave manipulator in the image pointing toward the distal portion of the slave manipulator in the image.

10. The system of claim 9, the means for transforming the image comprising means for rotating the image.

11. The system of claim 9, further comprising a second endoscope for capturing a stereo image of the at least one slave manipulator, the means for transforming the image on the display comprising means for transforming a stereo image of the display.

12. The system of claim 9, the distal portion of the slave manipulator coupled to a surgical end effector, the system further comprising:

means for positioning the surgical end effector within an internal surgical site of a patient; and

means for positioning the endoscope within the internal surgical site.

13. The system of claim 9, further comprising a second slave manipulator and a second master manipulator, the second slave manipulator having a distal portion and a proximal portion, the system further comprising:

means for moving the second slave manipulator in response to movement of the second master manipulator;

the display further displaying the image of the second slave manipulator to the operator.

14. The system of claim 9, the means for transforming comprising means for effecting dynamic synchronization of the position of the distal portion of the slave manipulator in the image with the position of the master manipulator.

15. The system of claim 9, the means for transforming comprising means for bringing the position of the slave manipulator in the image in registration with the position of the master manipulator.

16. The system of claim 9, the means for transforming comprising means for rotating the endoscope about its visual axis.