System and method for augmentation of endoscopic surgery

• US 5,417,210 A
• Filed: 05/27/1992
• Issued: 05/23/1995
• Est. Priority Date: 05/27/1992
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Taylor, Russell H., Funda, Janez, LaRose, David A.
• Primary Examiner(s): NOT, DEFINED
• Application Number:
  US07/889,215
• Time in Patent Office: 1,091 Days
• Field of Search: 128/653.1, 128/653.2, 128/653.5, 128/660.07, 128/662.06, 128/916, 128/4, 364/413.13, 606/14-17, 348/65
• US Class Current: 600/425
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: A61B 1/05 characterised by the image sensor, e.g. camera, being in the distal end portion
• CPC Class Codes: A61B 17/00234 for minimally invasive surgery A61B17/0218, A61B17/0469, A61B17/12013, A61B17/1285, A61B17/29, A61B17/320016 take precedence
• CPC Class Codes: A61B 2017/00022 Sensing or detecting at the treatment site
• CPC Class Codes: A61B 2017/00203 with speech control or speech recognition
• CPC Class Codes: A61B 2034/102 Modelling of surgical devices, implants or prosthesis
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/2068 using pointers, e.g. pointers having reference marks for determining coordinates of body points
• CPC Class Codes: A61B 2034/742 Joysticks
• CPC Class Codes: A61B 2090/0813 Accessories designed for easy sterilising, i.e. re-usable
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/3937 Visible markers
• CPC Class Codes: A61B 2090/3945 Active visible markers, e.g. light emitting diodes
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/70 Manipulators specially adapted for use in surgery
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 90/10 for stereotaxic surgery, e.g. frame-based stereotaxis
• CPC Class Codes: A61B 90/11 : with guides for needles or instruments, e.g. arcuate slides or ball joints
• CPC Class Codes: A61B 90/36 : Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/361 : Image-producing devices, e.g. surgical cameras
• CPC Class Codes: B25J 17/0275 : Universal joints, e.g. Hooke, Cardan, ball joints
• CPC Class Codes: B25J 9/04 : by rotating at least one arm, excluding the head movement itself, e.g. cylindrical coordinate type or polar coordinate type
• CPC Class Codes: G05B 2219/37074 : Projection device, monitor, track tool, workpiece form, process on display
• CPC Class Codes: G05B 2219/40418 : Presurgical planning, on screen indicate regions to be operated on
• CPC Class Codes: G05B 2219/40609 : Camera to monitor end effector as well as object to be handled
• CPC Class Codes: G05B 2219/45123 : Electrogoniometer, neuronavigator, medical robot used by surgeon to operate
• CPC Class Codes: G05B 2219/45166 : Tomography

Abstract

The present method and apparatus use image processing to determine information about the position of a designated object. The invention is particularly useful in applications where the object is difficult to view or locate. In particular, the invention is used in endoscopic surgery to determine positional information about an anatomical feature within a patient'"'"'s body. The positional information is then used to position or reposition an instrument (surgical instrument) in relation to the designated object (anatomical feature). The invention comprises an instrument which is placed in relation to the designated object and which is capable of sending information about the object to a computer. Image processing methods are used to generated images of the object and determine positional information about it. This information can be used as input to robotic devices or can be rendered, in various ways (video graphics, speech synthesis), to a human user. Various input apparatus are attached to the transmitting or other used instruments to provide control inputs to the computer.

Claims

1. A method determining positional information about an anatomical feature within a patient'"'"'s body comprising the steps of:

inserting a first surgical instrument into the patient'"'"'s body, the instrument having a means for transmitting an image out of the patient'"'"'s body;

designating an anatomical feature of interest by pointing with a second surgical instrument, one or more visual targets being on the second instrument;

transmitting an image of the designated anatomical feature out of the patient'"'"'s body using the first surgical instrument;

determining positional information about the designated anatomical feature of interest by using image processing.

2. A method of determining positional information about an anatomical feature within a patient'"'"'s body, comprising the steps of:

inserting a first surgical instrument into the patient'"'"'s body, the instrument having a means for transmitting an image out of the patient'"'"'s body;

transmitting an image of the designated anatomical feature out of the patient'"'"'s body using the first surgical instrument;

displaying the transmitted image;

designating an anatomical feature of interest by manipulating a computer generated graphics object displayed and superimposed on the display image transmitted by the first surgical instrument; and

determining positional information about the designated anatomical feature of interest by using image processing.

3. A method of determining positional information about an anatomical feature, as in claim 2, where the computer generated graphics object is manipulated by the use of a joystick mounted on the first surgical instrument.

4. A method of determining positional information about an anatomical feature, as in claim 2, where the computer generated graphics object is manipulated by the use of a force sensing device mounted on the first surgical instrument.

5. A method of determining positional information about an anatomical feature within a patient'"'"'s body, comprising the steps of:

inserting a first surgical instrument into the patient'"'"'s body, the instrument having a means for transmitting an image out of the patient'"'"'s body;

transmitting an image of the designated anatomical feature out of the patient'"'"'s body using the first surgical instrument;

designating an anatomical feature of interest by using either one of the following steps (1) and (2);

(1) pointing with a second surgical instrument having a visual target and

(2) manipulating a computer generated graphics object displayed and superimposed on the image transmitted by the first surgical instrument; and

further comprising the steps of;

determining positional information about the designated anatomical feature of interest by using image processing; and

providing the positional, information to a surgeon.

6. A method of determining positional information about an anatomical feature, as in claim 5, where the information is provided to the surgeon in the form of synthesized speech.

7. A method of determining positional information about an anatomical feature, as in claim 5, where the information is provided to the surgeon in the form of tactile feedback.

8. A method of determining positional information about an anatomical feature, as in claim 7, where the tactile feedback is provided in the form of vibrations on one of the surgical instruments held by the surgeon.

9. A method of determining positional information about an anatomical feature, as in claim 5, where the information is provided to the surgeon in the form of a computer generated graphics object superimposed on the image obtained from the first surgical instrument.

10. A method of determining positional information about an anatomical feature, as in claim 9, where the computer graphics object is displayed in two dimensions.

11. A method of determining positional information about an anatomical feature, as in claim 9, where the computer graphics object is displayed in three dimensions.

12. A method of determining positional information about an anatomical feature comprising:

inserting a first and a second surgical instrument into the patient'"'"'s body, each instrument having a means for transmitting an image out of the patient'"'"'s body;

obtaining a first image of the feature from a first vantage point using the first surgical instrument;

obtaining a second image of the feature using the second surgical instrument from a second vantage point, the second vantage point being at a known position and orientation with respect to the first vantage point;

locating the anatomical feature in both images;

computing the position of the anatomical feature relative to the vantage points using positional information about the feature in each image, together with the known position and orientation of the two vantage points with respect to each other.

13. A method of determining positional information about an anatomical feature, as in claim 12, where the anatomical feature is located in at least one of the images by computer image processing.

14. A method of determining positional information about an anatomical feature, as in claim 12, where the first vantage point is the position of the first surgical instrument, being one lens of a stereoscopic camera, and the second vantage point is the position of the second surgical instrument, being the position of the second lens of a stereoscopic camera.

15. A method of determining positional information about an anatomical feature comprising the steps of:

inserting a first surgical instrument into the patient'"'"'s body, the instrument having a means for transmitting an image out of the patient'"'"'s body;

obtaining a first image of the feature from a first vantage point using the first surgical instrument;

obtaining a second image of the feature using the first surgical instrument from a second vantage point, the second vantage point being at a known position and orientation with respect to the first vantage point;

locating the anatomical feature in the first and second images; and

computing the position of the anatomical feature relative to the first and second vantage points using positional information about the feature in each of the first and second images together with the known position and orientation of the two vantage points with respect to each other.