Stereo imaging system and method for use in telerobotic systems

• US 7,277,120 B2
• Filed: 03/07/2004
• Issued: 10/02/2007
• Est. Priority Date: 12/08/1998
• Status: Expired due to Term

• Current Assignee: Intuitive Surgical Operations, Inc. (Intuitive Surgical Holdings, LLC)
• Original Assignee: Intuitive Surgical Incorporated (Intuitive Surgical Holdings, LLC)
• Inventors: Stern, John D., Tierney, Michael J., Gere, David, Burns, Christopher R.
• Primary Examiner(s): Philippe; Gims
• Assistant Examiner(s): REKSTAD, ERICK J
• Application Number:
  US10/795,963
• Publication Number:
  US 20060092273A1
• Time in Patent Office: 1,304 Days
• Field of Search: 348/42, 348/51, 359/462, 382/154, 382/294
• US Class Current: 348/45
• CPC Class Codes: A61B 1/00183 for variable viewing angles
• CPC Class Codes: A61B 1/00193 adapted for stereoscopic vision
• CPC Class Codes: G02B 23/2415 Stereoscopic endoscopes
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/327 Calibration thereof
• CPC Class Codes: H04N 23/555 for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes

Abstract

This invention relates to a stereo imaging system for use in telerobotic systems. A method of imaging a target site in a stereo imaging system is provided. The method typically includes capturing a right and a left optical image of the target site and transforming the right and the left optical images preferably into digital information. The method further includes converting the digital information into opposed images of the target site displayed on a stereo display of the stereo imaging system, one of the opposed images being associated with the right optical image and the other of the opposed images being associated with the left optical image. The method further includes regulating the digital information to cause the positions of the target site displayed on the opposed images to change relative to each other. The invention further provides for a method of aligning the opposed images, to a method of adjusting the stereo working distance of an image capture device, such as an endoscope and to a stereo imaging system.

Claims

1. A method of aligning left and right stereoscopic images, the method comprising:

capturing right stereoscopic image pixel information for a target object in a first array;

capturing left stereoscopic image pixel information for the target object in a second array;

calculating a first intersection position in the first array by;

for each of a plurality of rows in the first array, determining a first pixel column position that exceeds a row threshold value, a last pixel column position that exceeds the row threshold value, and a column position between the first pixel column position and the last pixel column position;

for each of a plurality of columns in the first array, determining a first pixel row position that exceeds a column threshold value, a last pixel row position that exceeds the column threshold value, and row position between the first pixel row position and the last pixel row position; and

determining a position at which a first line fitted to the column positions between the first pixel column positions and the last pixel column positions intersects a second line fitted to the row positions between the first pixel row positions and the last pixel row positions;

calculating a second intersection position in the second array;

selecting a portion of the first array and a portion of the second array such that the calculated intersection positions for each array substantially occupy the same position relative to the selected portions; and

outputting an aligned stereoscopic image to a viewer by displaying the selected portion of the first array and the selected portion of the second array.

2. The method of claim 1, wherein the column position between the first pixel column position and the last pixel column position is based on a mean value of pixel values between the first pixel column position and the last pixel column position.

3. The method of claim 1, wherein:

the plurality of rows comprises all rows in the array;

or

the plurality of columns comprises all columns in the array;

or

the plurality of rows comprises all rows in the array and the plurality of columns comprises all columns in the array.

4. The method of claim 1, wherein:

the column position between the first pixel column position and the last pixel column position is based on a mean value of pixel values between the first pixel column position and the last pixel column position;

or

the row position between the first pixel row position and the last pixel row position is based on a mean value of pixel values between the first pixel row position and the last pixel row position;

or

the column position between the first pixel column position and the last pixel column position is based on a mean value of pixel values between the first pixel column position and the last pixel column position, and the row position between the first pixel row position and the last pixel row position is based on a mean value of pixel values between the first pixel row position and the last pixel row position.

5. A method of aligning left and right stereoscopic images, the method comprising:

capturing right stereoscopic image pixel information for a target object at a surgical site in a first array;

capturing left stereoscopic image pixel information for the target object in a second array;

calculating a first line position in the first array;

calculating a second line position in the second array;

selecting a portion of the first array and a portion of the second array such that the calculated line positions for each array substantially occupy the same position relative to the selected portions;

outputting an aligned stereoscopic image to a viewer by displaying the selected portion of the first array and the selected portion of the second array; and

maintaining a selected working distance of an endoscope used to capture the right and left stereoscopic image pixel information by periodically repeating the acts of capturing right stereoscopic image pixel information, capturing left stereoscopic image pixel information, calculating a first line position, calculating a second line position, selecting a portion of the first array and a portion of the second array, and outputting an aligned stereoscopic image.

6. The method of claim 5, wherein calculating the first line position comprises:

for each of a plurality of rows in the first array, determining a first pixel column position that exceeds a row threshold value, a last pixel column position that exceeds the row threshold value, and a column position between the first pixel column position and the last pixel column position; and

fitting the first line to the column positions between the first pixel column positions and the last pixel column positions.

7. The method of claim 6, wherein the column position between the first pixel column position and the last pixel column position is based on a mean value of pixel values between the first pixel column position and the last pixel column position.

8. The method of claim 6, wherein the plurality of rows comprises all rows in the array.

9. The method of claim 5, wherein calculating a first line position comprises using a pattern matching template for the first array, and wherein the first array pixel information corresponds to an image of a surgical site.

10. An endoscopic imaging system comprising:

an image capture stage that captures right stereoscopic image pixel information for a target object in a first array and left stereoscopic image pixel information for the target object in a second array;

a video processing stage that

calculates a first intersection position in the first array and a second intersection position in the second array, wherein calculating the first intersection position comprises;

for each of a plurality of rows in the first array, determining a first pixel column position that exceeds a row threshold value, a last pixel column position that exceeds the row threshold value, and a column position between the first pixel column position and the last pixel column position;

for each of a plurality of columns in the first array, determining a first pixel row position that exceeds a column threshold value, a last pixel row position that exceeds the column threshold value, and row position between the first pixel row position and the last pixel row position; and

determining a position at which a first line fitted to the column positions between the first pixel column positions and the last pixel column positions intersects a second line fitted to the row positions between the first pixel row positions and the last pixel row positions, and

selects a portion of the first array and a portion of the second array such that the calculated intersection positions for each array substantially occupy the same position relative to the selected portions; and

a monitor stage that outputs an aligned stereoscopic image to a viewer by displaying the selected portion of the first array and the selected portion of the second array.

11. The system of claim 10, wherein the target object comprises an object at a surgical site on which a surgical procedure is performed.

12. The system of claim 10, wherein the column position between the first pixel column position and the last pixel column position is based on a mean value of pixel values between the first pixel column position and the last pixel column position.

13. The system of claim 10, wherein:

the plurality of rows comprises all rows in the array;

or

the plurality of columns comprises all columns in the array;

or

the plurality of rows comprises all rows in the array and the plurality of columns comprises all columns in the array.

14. The system of claim 10, wherein:

the column position between the first pixel column position and the last pixel column position is based on a mean value of pixel values between the first pixel column position and the last pixel column position;

or

the row position between the first pixel row position and the last pixel row position is based on a mean value of pixel values between the first pixel row position and the last pixel row position;

or

the column position between the first pixel column position and the last pixel column position is based on a mean value of pixel values between the first pixel column position and the last pixel column position and the row position between the first pixel row position and the last pixel row position is based on a mean value of pixel values between the first pixel row position and the last pixel row position.

15. The system of claim 10, wherein calculating a first intersection position comprises using a pattern matching template for the first array, and wherein the first array pixel information corresponds to an image of a surgical site.

16. The system of claim 10, wherein the acts of capturing right stereoscopic image pixel information, capturing left stereoscopic image pixel information, calculating a first intersection position, calculating a second intersection position, selecting a portion of the first array and a portion of the second array, and outputting an aligned stereoscopic image are automatically repeated to compensate for stereoscopic image misalignment.

17. The system of claim 10 further comprising an endoscope through which the right stereoscopic image and the left stereoscopic image are transmitted to the image capture stage.