IMAGE INTEGRATION BASED REGISTRATION AND NAVIGATION FOR ENDOSCOPIC SURGERY

• US 20120294498A1
• Filed: 12/06/2010
• Published: 11/22/2012
• Est. Priority Date: 01/13/2010
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Popovic, Aleksandra
• Granted Patent: US 9,095,252 B2
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/042 characterised by a proximal camera, e.g. a CCD camera
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A real-time tracking of surgical tools relative to a pre-operative surgical plan and intra-operative images involves an image-based registration and tool tracking registration. The image-based registration implements an integration of a plurality of images of an anatomical region of a body including a pre-operative scan image (31) (e.g., a 3D CT/MRI image), an intra-operative fluoroscopic image (42) (e.g., an 2D X-ray image) and an intra- operative endoscopic image (23) (e.g., an 2D arthroscopic image). The tool tracking registration implements a representation within the pre-operative scan image (31) and/or the intra-operative fluoroscopic image (42) of a tracking of one or more surgical tools within the intra-operative endoscopic image (23).

Claims

1. A surgical navigation system (60) for integrating a plurality of images of an anatomical region of a body including a pre-operative scan image (31), an intra-operative fluoroscopic image (42) and an intra-operative endoscopic image (23), the surgical navigation system (60) comprising:

an image integrator (61) operable to generate an integrated image matrix (T_X-E) including an integration of a fluoroscopic image matrix (T_X-CT) and an endoscopic image matrix (T_CT-E),

wherein the fluoroscopic image matrix (T_X-CT) includes a transformation between the intra-operative fluoroscopic image (42) and the pre-operative scan image (31), and

wherein the endoscopic image matrix (T_CT-E) includes a transformation between the pre-operative scan image (31) and the intra-operative endoscopic image (23); and

a tool tracker (62) operable to generate an integrated tracking matrix (T_T-X) and a scan tracking matrix (T_T-CT),

wherein the integrated tracking matrix (T_T-X) includes an integration of the integrated image matrix (T_X-E) and an endoscopic tracking matrix (T_T-E),

wherein the scan tracking matrix (T_T-CT) includes an integration of the endoscopic image matrix (T_CT-E) and the endoscopic tracking matrix (T_T-E), and

wherein the endoscopic tracking matrix (T_T-E) represents a tracking of a surgical tool (21) within the intra-operative endoscopic image (23).

2. The surgical navigation system (60) of claim 1, wherein the image integrator (61) is further operable to execute a photogrammetric reconstruction of a surface of an object within the intra-operative endoscopic image (23) as a basis for the transformation between the pre-operative scan image (31) and the intra-operative endoscopic image (23).

3. The surgical navigation system (60) of claim 1, wherein the integration of the fluoroscopic image matrix (T_X-CT) and the endoscopic image matrix (T_CT-E) includes an execution of a matrix multiplication involving the fluoroscopic image matrix (T_X-CT) and the endoscopic image matrix (T_CT-E).

4. The surgical navigation system (60) of claim 1, wherein the integration of the integrated image matrix (T_X-E) and an endoscopic tracking matrix (T_T-E) includes an execution of a matrix multiplication involving the integrated image matrix (T_X-E) and the endoscopic tracking matrix (T_T-E).

5. The surgical navigation system (60) of claim 1, wherein the integration of the endoscopic image matrix (T_CT-E) and the endoscopic tracking matrix (T_T-E) includes an execution of a matrix multiplication involving the endoscopic image matrix (T_CT-E) and the endoscopic tracking matrix (T_T-E).

6. The surgical navigation system (60) of claim 1, further comprising:

a display device for displaying the intra-operative fluoroscopic image (42) having an overlay (43) of a surgical tool (21) as a function of the integrated tracking matrix (T_T-X).

7. The surgical navigation system (60) of claim 1, further comprising:

a display device for displaying the pre-operative scan image (31) having an overlay (43) of a surgical tool (21) as a function of the scan tracking matrix (T_T-CT).

8. The surgical navigation system (60) of claim 1, wherein the tool tracker (62) is further operable to execute a photogrammetric tracking of the surgical tool (21) within the intra-operative endoscopic image (23).

9. The surgical navigation system (60) of claim 1, wherein the pre-operative scan image (31) is selected from a group consisting of a computed tomography image, magnetic resonance image, an ultrasound image and an X-ray image.

10. A surgical navigation system (60) for integrating a plurality of images of an anatomical region of a body including a pre-operative scan image (31), an intra-operative fluoroscopic image (42) and an intra-operative endoscopic image (23), the surgical navigation system (60) comprising:

an image integrator (61) operable to generate an integrated image matrix (T_X-E) including an integration of a fluoroscopic image matrix (T_X-CT) and an endoscopic image matrix (T_CT-E),

wherein the fluoroscopic image matrix (T_X-CT) includes a transformation between the intra-operative fluoroscopic image (42) and the pre-operative scan image (31), and

wherein the endoscopic image matrix (T_CT-E) includes a transformation between the pre-operative scan image (31) and the intra-operative endoscopic image (23); and

a tool tracker (62) operable to generate an integrated tracking matrix (T_T-X),

wherein the integrated tracking matrix (T_T-X) includes an integration of the integrated image matrix (T_X-E) and an endoscopic tracking matrix (T_T-E), and

wherein the endoscopic tracking matrix (T_T-E) represents a tracking of a surgical tool (21) within the intra-operative endoscopic image (23).

11. The surgical navigation system (60) of claim 10, the image integrator (61) is further operable to reconstruct a surface of an object within the intra-operative endoscopic image (23) as a basis for the transformation between the pre-operative scan image (31) and the intra-operative endoscopic image (23).

12. The surgical navigation system (60) of claim 10, wherein the integration of the fluoroscopic image matrix (T_X-CT) and the endoscopic image matrix (T_CT-E) includes an execution of a matrix multiplication involving the fluoroscopic image matrix (T_X-CT) and the endoscopic image matrix (T_CT-E).

13. The surgical navigation system (60) of claim 10, wherein the integration of the integrated image matrix (T_X-E) and an endoscopic tracking matrix (T_T-E) includes an execution of a matrix multiplication involving the integrated image matrix (T_X-E) and the endoscopic tracking matrix (T_T-E).

14. The surgical navigation system (60) of claim 10, further comprising:

a display device for displaying the intra-operative fluoroscopic image (42) having an overlay (43) of a surgical tool (21) as a function of the integrated tracking matrix (T_T-X).

15. The surgical navigation system (60) of claim 10, wherein the tool tracker (62) is further operable to execute a photogrammetric tracking of the surgical tool (21) within the intra-operative endoscopic image (23).

16. A surgical navigation method for integrating a plurality of images of an anatomical region of a body including a pre-operative scan image (31), an intra-operative fluoroscopic image (42) and an intra-operative endoscopic image (23), comprising:

generating an integrated image matrix (T_X-E) including an integration of a fluoroscopic image matrix (T_X-CT) and an endoscopic image matrix (T_CT-E),

wherein the fluoroscopic image matrix (T_X-CT) includes a transformation between the intra-operative fluoroscopic image (42) and the pre-operative scan image (31), and

wherein the endoscopic image matrix (T_CT-E) includes a transformation between the pre-operative scan image (31) and the intra-operative endoscopic image (23); and

generating an integrated tracking matrix (T_T-X) and a scan tracking matrix (T_T-CT),

wherein the integrated tracking matrix (T_T-X) includes an integration of the integrated image matrix (T_X-E) and an endoscopic tracking matrix (T_T-E),

wherein the scan tracking matrix (T_T-CT) includes an integration of the endoscopic image matrix (T_CT-E) and the endoscopic tracking matrix (T_T-E), and

wherein the endoscopic tracking matrix (T_T-E) represents a tracking of a surgical tool (21) within the intra-operative endoscopic image (23).

17. The surgical navigation method of claim 16, wherein the integration of the fluoroscopic image matrix (T_X-CT) and the endoscopic image matrix (T_CT-E) includes an execution of a matrix multiplication involving the fluoroscopic image matrix (T_X-CT) and the endoscopic image matrix (T_CT-E).

18. The surgical navigation method of claim 16, wherein the integration of the integrated image matrix (T_X-E) and an endoscopic tracking matrix (T_T-E) includes an execution of a matrix multiplication involving the integrated image matrix (T_X-E) and the endoscopic tracking matrix (T_T-E).

19. The surgical navigation method of claim 16, wherein the integration of the endoscopic image matrix (T_CT-E) and the endoscopic tracking matrix (T_T-E) includes an execution of a matrix multiplication involving the endoscopic image matrix (T_CT-E) and the endoscopic tracking matrix (T_T-E).

20. The surgical navigation method of claim 16, further comprising:

displaying the intra-operative fluoroscopic image (42) having an overlay (43) of a surgical tool (21) as a function of the integrated tracking matrix (T_T-X); and

displaying the pre-operative scan image (31) having an overly (43) of a surgical tool (21) as a function of the scan tracking matrix (T_T-CT).