Camera calibration for endoscope navigation system

• US 7,892,165 B2
• Filed: 10/23/2006
• Issued: 02/22/2011
• Est. Priority Date: 10/23/2006
• Status: Active Grant

• Current Assignee: Hoya Corporation
• Original Assignee: Hoya Corporation
• Inventors: Nakamura, Tetsuya
• Primary Examiner(s): Leubecker; John P
• Assistant Examiner(s): CHANG, JEFFREY HAO-WEI
• Application Number:
  US11/551,927
• Publication Number:
  US 20080097156A1
• Time in Patent Office: 1,583 Days
• Field of Search: 600/109, 600/117, 600/424, 600/426, 600/427, 600/429, 600/473, 606/130, 73/17.9
• US Class Current: 600/117
• CPC Class Codes: A61B 1/045 Control thereof
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/745 using a holographic display
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras

Abstract

A scope navigation apparatus, method and a program embodied computer readable medium for navigating a scope end that is to be inserted into an object. The scope end includes at least one sensor that senses a spatial location of the scope end. An image pickup device records one or more images of a calibration device, which includes a sensor that senses a location of the calibration device. A processor calibrates the images recorded by the image pickup device and corrects for intrinsic and extrinsic parameters using a calibration matrix.

Claims

1. A scope navigation apparatus comprising:

a scope configured to be inserted into an object, the scope including at least one sensor;

a calibration device configured to calibrate the scope;

a calibration device sensor positioned in a fixed position in relation to the calibration device;

an image pickup device that records at least one image, the at least one image being an image of the calibration device;

an axis determiner that determines an optical axis of the image pickup device, a central axis of the at least one sensor being offset from the optical axis of the image pickup device;

a location determiner that senses a location of the at least one sensor and a location of the calibration device sensor; and

a processor that corrects the at least one image recorded by the image pickup device based upon the location of the at least one sensor and the location of the calibration device sensor, said processor being configured to compensate for the offset between the central axis of the at least one sensor and the optical axis of the image pickup device.

2. The apparatus according to claim 1, the processor comprising:

a video processor; and

a host device, where at least one of the video processor and host device corrects the at least one image recorded by the image pickup device.

3. The apparatus according to claim 1, the location determiner comprising:

a sensor interface unit; and

a system control unit, wherein the sensor interface unit senses the location of the at least one sensor and the location of the calibration device sensor.

4. The apparatus according to claim 1, further comprising a field generator that generates an electromagnetic field, wherein said location determiner senses the location of the at least one sensor and the location of the calibration device sensor based on the electromagnetic field.

5. The apparatus according to claim 4, wherein the calibration device is positioned in a fixed relation to the field generator.

6. The apparatus according to claim 1, wherein the at least one sensor is inserted into a channel of the scope, and said calibration device sensor is affixed to the calibration device.

7. The apparatus according to claim 1, wherein the calibration device is at least one of a checkerboard pattern, a dot pattern, or a grid pattern.

8. A method for calibrating and correcting an image to be displayed, the method comprising:

recording an image of a calibration device, the calibration device located in a fixed position in relation to at least one calibration device sensor;

determining an optical axis of an image pickup device, the image pickup device including at least one sensor;

detecting a location of the at the least one calibration device sensor and a location of the at least one sensor;

determining a parameter value for calibrating the image pickup device;

compensating an offset between a central axis of the at least one sensor and the optical axis; and

calibrating the recorded image based on the parameter value.

9. The method according to claim 8, said determining a parameter value comprising:

selecting a first point on the recorded image;

calculating a first determination point based on the selected first point;

determining a difference value between the selected point and the first determination point; and

comparing the difference value with a threshold value.

10. The method according to claim 9, further comprising:

determining the difference value to be greater than, or equal to the threshold value; and

calculating a calibration matrix for calibrating the image pickup device.

11. The method according to claim 10, wherein the calibration matrix includes at least one intrinsic parameter and at least one extrinsic parameter.

12. The method according to claim 11, the calibration matrix being based on a relationship between a three dimensional coordinate system of the calibration device and a two dimensional coordinate system of the image pickup device.

13. The method according to claim 9, wherein said calibrating is based on at least one of five degrees of freedom and six degrees of freedom.

14. The method according to claim 9, further comprising:

setting the first point as an origin point; and

selecting a second point in at least one of a clockwise and counterclockwise direction.

15. The method for calibrating according to claim 8, further comprising providing at least one of a checkerboard pattern, a dot pattern, or a grid pattern, as the calibration device.

16. A computer readable medium comprising:

an image recording code segment that causes recording an image of a calibration device, the calibration device located in a fixed position in relation to at least one calibration device sensor;

a determining code segment that causes determining an optical axis of an image pickup device, the image pickup device including at least one sensor;

a detecting code segment that causes detecting a location of the at the least one sensor and a location of the at least one calibration device sensor;

a parameter determining code segment that causes determining a parameter value for calibrating the image pickup device;

a compensating code segment that causes compensating an offset between a central axis of the at least one sensor and the optical axis of the image pickup device; and

a calibrating code segment that causes calibrating the image pickup device based on the parameter value.

17. The computer readable medium according to claim 16, the determining code segment comprising:

a point selecting code segment that causes selecting a first point on the recorded image;

a determination point calculating code segment that causes calculating a first determination point based on the selected first point;

a difference value determining code segment that causes determining a value between the selected point and the first determination point; and

a difference value comparing code segment that causes comparing the difference value with a threshold value.

18. The computer readable medium according to claim 16, further comprising:

a difference value determining code segment that causes determining the difference value to be greater than, or equal to the threshold value; and

a calibration matrix code segment that causes calculating a calibration matrix for calibrating the image pickup device.

19. The computer readable medium according to claim 18, wherein the calibrating is based on at least one of five degrees of freedom and six degrees of freedom, and wherein the calibration matrix includes at least one intrinsic parameter and at least one extrinsic parameter.

20. The computer readable medium according to claim 18, the calibration matrix being based on a relationship between a three dimensional coordinate system of the calibration device and a two dimensional coordinate system of the image pickup device.