Multi-dimensional navigation of endoscopic video

• US 8,248,414 B2
• Filed: 09/18/2006
• Issued: 08/21/2012
• Est. Priority Date: 09/18/2006
• Status: Active Grant

• Current Assignee: Stryker Corporation
• Original Assignee: Stryker Corporation
• Inventors: Gattani, Abhishek, Hameed, Salmaan
• Primary Examiner(s): Martello, Edward
• Application Number:
  US11/523,217
• Publication Number:
  US 20080071143A1
• Time in Patent Office: 2,164 Days
• Field of Search: 600/117, 345/424
• US Class Current: 345/424
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/009 with bending or curvature detection of the insertion part
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2061 using shape-sensors, e.g. fiber shape sensors with Bragg gratings
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/064 using markers A61B5/062 takes precedence
• CPC Class Codes: G06T 2207/10068 Endoscopic image
• CPC Class Codes: G06T 2207/30021 Catheter; Guide wire
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

An endoscopic surgical navigation system comprises a multi-dimensional video generation module that enables a user to visually navigate captured endoscopic video with six degrees of freedom. This capability provides the user with control of a virtual camera (point of view) that can be translated in three orthogonal axes in 3-D space as well as allowing control of vertical panning (pitch), horizontal panning (yaw) and tilt (roll) of the virtual camera, as well as zoom.

Claims

1. A computer-implemented method comprising:

acquiring, by the computer, video frames of an anatomical region of interest from a video camera optically coupled to an endoscope during an endoscopic procedure, the video frames being acquired in a sequence of instances of time;

determining, by the computer, a position and orientation of a distal tip of the endoscope for each of the video frames as the video frames are acquired;

associating, by the computer, each of the video frames with data indicative of the position and orientation of the distal tip as the video frames were acquired; and

processing, by the computer, the video frames to produce a plurality of processed frames, to enable a user to visually navigate the video frames of the anatomical region of interest during the endoscopic procedure out of sequence using the position and orientation data associated with the processed frames, and with at least one degree of freedom that is not represented in the sequence of instances of time of the video frames acquired from the camera.

2. A method as recited in claim 1, wherein said processing the video frames comprises:

processing the video frames to enable the user to visually navigate the anatomical region of interest with six degrees of freedom.

3. A method as recited in claim 1, wherein said processing the video frames comprises:

processing the video frames to enable the user to visually navigate the anatomical region of interest with control of a virtual endoscope point of view using spatial translation in each of three dimensions and yaw, pitch and roll.

4. A method as recited in claim 1,

wherein said processing the video frames comprises:

computing a position and orientation of an image plane for each of the video frames as a function of the corresponding position and orientation of the distal tip of the endoscope; and

transforming the image planes for the frames to user-specified spatial positions and orientations, in response to user inputs.

5. A method as recited in claim 4, further comprising:

applying interpolation to smooth transitions between intersecting image planes.

6. A computer-implemented method comprising:

acquiring, by the computer, video frames of an anatomical region of interest from a video camera optically coupled to a flexible endoscope during an endoscopic procedure, the video frames being acquired in a sequence of instances of time;

determining, by the computer, a position and orientation of a distal tip of the flexible endoscope for each of the video frames as the video frames are acquired;

associating, by the computer, in memory, with each of the video frames, data indicative of the position and orientation of the distal tip as the video frames were acquired; and

processing, by the computer, the video frames to produce a plurality of processed frames, to enable a user to visually navigate the video frames of the anatomical region of interest during the endoscopic procedure out of sequence using the position and orientation data associated with the processed frames by controlling a virtual endoscope point of view with spatial translation in each of three dimensions and yaw, pitch and roll.

7. A method as recited in claim 6, wherein said processing comprises:

computing a position and orientation of an image plane for each of the video frames as a function of the corresponding position and orientation of the distal tip of the endoscope; and

in response to user input specifying a visual navigation action, transforming the image plane of one of the frames to a new spatial position or orientation based on the user input.

8. A method as recited in claim 7, further comprising:

causing said one of the frames to be displayed according to said transformed image plane.

9. A method as recited in claim 6, further comprising:

using interpolation to smooth transitions between intersecting image planes.

10. A system comprising:

a processor; and

a memory coupled with the processor and storing instructions which, when executed by the processor, cause the processor to

acquire video frames of an anatomical region of interest from a video camera that is optically coupled to an endoscope during an endoscopic procedure, the video frames being acquired in a sequence of instances of time;

determine a position and orientation of a distal tip of the endoscope for each of the video frames as the video frames are acquired;

associate each of the video frames with data indicative of the position and orientation of the distal tip as the video frames were acquired; and

process the video frames to produce a plurality of processed frames, to enable a user to visually navigate the video frames of the anatomical region of interest during the endoscopic procedure out of sequence using the position and orientation data associated with the processed frames, and with at least one degree of freedom that is not represented in the sequence of instances of time of the video frames acquired from the camera.

11. A system as recited in claim 10, further comprising a user interface subsystem to receive user inputs for controlling visual navigation of the anatomical region of interest.

12. A system as recited in claim 10, wherein the video processing subsystem comprises:

logic to process the video frame data to produce the plurality of frames, to enable the user to visually navigate the anatomical region of interest with six degrees of freedom.

13. A system as recited in claim 10, wherein the video processing subsystem comprises:

logic to process the video frames to enable the user to visually navigate the anatomical region of interest with control of a virtual endoscope point of view using spatial translation in each of three dimensions and yaw, pitch and roll.

14. A system as recited in claim 13,

wherein the video processing subsystem comprises:

logic to compute a position and orientation of an image plane for each of the video frames as a function of the corresponding position and orientation of the distal tip of the endoscope; and

logic to transform the image planes for the frames to user-specified spatial positions and orientations, in response to user inputs.

15. A system as recited in claim 14, further comprising logic to apply interpolation to smooth transitions between intersecting image planes.

16. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processing system to perform a method comprising:

acquiring video frames of an anatomical region of interest from a video camera optically coupled to an endoscope during an endoscopic procedure, the video frames being acquired in a sequence of instances of time;

determining a position and orientation of a distal tip of the endoscope for each of the video frames as the video frames are acquired;

associating each of the video frames with data indicative of the position and orientation of the distal tip as the video frames were acquired; and

processing the video frames to produce a plurality of processed frames, to enable a user to visually navigate the video frames of the anatomical region of interest during the endoscopic procedure out of sequence using the position and orientation data associated with the processed frames, and with at least one degree of freedom that is not represented in the sequence of instances of time of the video frames acquired from the camera.

17. The non-transitory machine-readable storage medium as recited in claim 16, wherein said processing the video frames comprises:

computing a position and orientation of an image plane for each of the video frames as a function of the corresponding position and orientation of the distal tip of the endoscope; and

transforming the image planes for the frames to user-specified spatial positions and orientations, in response to user inputs.