Method and apparatus for displaying endoscopic images

• US 7,232,409 B2
• Filed: 11/20/2003
• Issued: 06/19/2007
• Est. Priority Date: 11/20/2003
• Status: Active Grant

• Current Assignee: Karl Storz Imaging, Inc (Karl Storz Se & Co. Kg)
• Original Assignee: KARL Storz Development Corp.
• Inventors: Hale, Eric Lawrence, Schara, Nathan Jon, Hoeg, Hans David
• Primary Examiner(s): Leubecker; John P
• Application Number:
  US10/718,434
• Publication Number:
  US 20050113643A1
• Time in Patent Office: 1,307 Days
• Field of Search: 600/101, 600/118, 600/117, 600/407, 600/476, 600/477, 382/128, 382/131, 345/419, 345/427
• US Class Current: 600/118
• CPC Class Codes: A61B 1/00009 of image signals during a use of endoscope
• CPC Class Codes: A61B 1/00045 Display arrangement
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations

Abstract

A method is disclosed for displaying endoscopic images that includes three-dimensional surfaces, variable viewing points, directions, and orientations. A computer receives a captured endoscopic image. A virtual surface is defined in the computer with the captured image textured onto the virtual surface. A virtual viewing point, virtual viewing direction, and virtual viewing orientation are defined relative to the virtual surface. A rendered image of the virtual surface is then created. The rendered image is displayed to the user. Video is displayed on the virtual surface by updating the image texture with each new frame.

Claims

1. A method for providing images of a viewed surface to a user, the method comprising:

moving an endoscope relative to a real surface which provides an endoscope view;

using the endoscope to acquire an image of the real surface;

providing a virtual surface approximating the topography of the real surface;

mapping the acquired image onto the virtual surface;

establishing a second viewing position relative to the real surface that represents a view different than the endoscope view;

determining position data indicating the difference between the endoscope view and the view from the second viewing position as the endoscope moves;

using the mapped virtual surface and the position data to render an image representing a view of the real surface from the second viewing position; and

providing the rendered image to the user.

2. The method of claim 1, wherein at least the steps of using the endoscope to acquire an image, mapping the acquired image onto the virtual surface, and using the mapped virtual surface and the position data to render an image are repeated when the endoscope is moved relative to the real surface.

3. The method of claim 1, wherein the image is one of a series of video images.

4. The method of claim 1, wherein the topographical approximation is based on volumetric scan data.

5. The method of claim 1, wherein the topographical approximation is based on stereo imaging.

6. The method of claim 1, wherein the second viewing position represents the position of a user.

7. The method of claim 1, wherein the virtual surface represents an anatomical object.

8. The method of claim 1, wherein the virtual surface is planar.

9. The method of claim 1, wherein the position of the endoscope is represented by a first viewing set including a scope viewing point, a scope viewing direction, and a scope orientation relative to the real surface, and the second position is represented by a second viewing set including a virtual viewing point, a virtual viewing direction, and a virtual orientation corresponding to the second position.

10. The method of claim 1, wherein a virtual viewing point is arranged in a manner generally corresponding to an endoscopic viewing point.

11. The method of claim 1, wherein a virtual viewing point is arranged in a manner generally corresponding to an actual viewing point of a user.

12. The method of claim 1, wherein a virtual viewing direction is directed in a manner generally corresponding to an actual viewing direction of a user.

13. The method of claim 1, wherein a virtual viewing orientation is oriented in a manner generally corresponding to an actual viewing orientation of a user.

14. The method of claim 1, wherein the image is mapped onto the virtual surface according to a mapping that adjusts for distortion.

15. An apparatus for providing images of a viewed surface to a user, comprising:

an endoscope providing an endoscope view that captures an image of a real surface when moved relative to the real surface;

a processor that creates a virtual surface approximating the topography of the real surface, maps the image acquired by said endoscope onto the virtual surface, determines position data indicating the difference between the endoscope view and a view from a second viewing position relative the real surface different than the endoscope view as the endoscope moves, and uses the mapped virtual surface and the position data to render an image representing a view of the real surface from the second viewing position; and

a monitor in communication with said computer that displays the rendered image.

16. The apparatus of claim 15, wherein, each time the endoscope acquires a new image, the processor maps the new image onto the virtual surface and uses the mapped virtual surface and the position data to render another image.

17. The method of claim 15 wherein the image is one of a series of video images.

18. A method for providing images of a viewed surface to a user, the method comprising:

inserting an endoscope into a cavity;

moving the endoscope relative to a real surface which provides an endoscope view;

using the endoscope to acquire an image of the real surface;

providing a virtual surface approximating the topography of the real surface;

mapping the acquired image onto the virtual surface;

establishing a second viewing position representing a view different than the endoscope view relative to the real surface;

determining position data indicating the difference between the endoscope view and the view from the second viewing position as the endoscope moves;

using the mapped virtual surface and the position data to render an image representing a view of the real surface from the second viewing position; and

providing the rendered image to the user.

19. The method of claim 18 wherein at least the steps of using the endoscope to acquire an image, mapping the acquired image onto the virtual surface, and using the mapped virtual surface and the position data to render an image are repeated when the endoscope is moved relative to the real surface.

20. The method of claim 18, wherein the image is one of a series of video images.