REAL-TIME DEPTH ESTIMATION FROM MONOCULAR ENDOSCOPE IMAGES

• US 20120056986A1
• Filed: 03/25/2010
• Published: 03/08/2012
• Est. Priority Date: 04/29/2009
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Popovic, Aleksandra
• Granted Patent: US 9,750,399 B2
• Time in Patent Office: Days
• US Class Current: 348/45
• CPC Class Codes: A61B 1/000094 extracting biological structures
• CPC Class Codes: A61B 1/04 combined with photographic or television appliances
• CPC Class Codes: A61B 1/313 for introducing through surgical openings, e.g. laparoscopes
• CPC Class Codes: A61B 2090/061 for measuring dimensions, e.g. length
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/065 Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe
• CPC Class Codes: A61B 8/461 Displaying means of special interest

Abstract

An endoscopic surgical method involves an advancement of an endoscope (20) to a target location within an anatomical region of a body and a generation of a plurality of endoscopic video frames (22) as the endoscope (20) is advanced to the target location with the endoscopic video frames (2) illustrating monocular endoscopic images of the anatomical region. For real-time estimation of a depth of an object within monocular endoscopic images (e.g., depth of a bronchial wall within an monocular endoscopic images of a bronchial tube), the method further involves (S41) a determination of an optical flow of one or more image points within a frame time series of the monocular endoscopic images of the anatomical region, and (S42) an estimation of a depth field indicative of a depth of the object within the monocular endoscopic images as a function of the optical flow of the image point(s).

Claims

1. A minimally invasive surgical system (10), comprising:

an endoscope (20) for generating a plurality of endoscopic video frames (22) as the endoscope (20) is advanced to a target location within an anatomical region of a body, the endoscopic video frames (22) illustrating monocular endoscopic images of the anatomical region; and

an endoscopic surgical control unit (30) in communication with the endoscope (20) to receive the endoscopic video frames (22) as the endoscope (20) is advanced to the target location,

wherein the endoscopic surgical control unit (30) is operable to estimate a depth field (36) indicative of a depth of an object within the monocular endoscopic images of the anatomical region as a function of an optical flow of at least one image point within a frame time series of the monocular endoscopic image of the anatomical region.

2. The minimally invasive surgical system (10) of claim 1, wherein the endoscopic surgical control unit (30) is further operable to generate a depth map display (37) representative of a depth field estimation.

3. The minimally invasive surgical system (10) of claim 1, wherein the endoscopic surgical control unit (30) is further operable to register the monocular endoscopic images with a pre-operative image of the anatomical region of the body as a function of the depth field estimation.

4. The minimally invasive surgical system (10) of claim 1, wherein the endoscopic surgical control unit (30) is further operable to pre-operatively plan a kinematic path (21) for the endoscope (20) to reach the target location within the anatomical region.

5. The minimally invasive surgical system (10) of claim 1, wherein the endoscopic surgical control unit (30) is further operable to track positions of the endoscope (20) within the anatomical region as the endoscope (20) is advanced to the target location within the anatomical region.

6. The minimally invasive surgical system (10) of claim 5, wherein a generation of the optical flow of the at least one image point within the frame time series of the monocular endoscopic images of the anatomical region includes a tracking of the positions of the endoscope (20) within the anatomical region.

7. The minimally invasive surgical system (10) of claim 1, wherein the endoscope (20) is one of a group including a bronchoscope and a nested cannula.

8. An endoscopic surgical method, comprising:

advancing an endoscope (20) to a target location within an anatomical region of a body;

generating a plurality of endoscopic video frames (22) as the endoscope (20) is advanced to the target location, the endoscopic video frames (22) illustrating a monocular endoscopic images of the anatomical region;

(S41) determining an optical flow of at least one image point within a frame time series of the monocular endoscopic images of the anatomical region; and

(S42) estimating a depth field indicative of a depth of an object within the monocular endoscopic images as a function of the optical flow of the at least one image point.

9. The endoscopic surgical method of claim 8, wherein the determination of the optical flow includes:

(S51) generating a vector field including a plurality of vectors, each vector indicative of a motion of one of the image points within the frame time series.

10. The endoscopic surgical method of claim 9, wherein the estimation of the depth field includes:

(S52) identifying a focus of expansion within the vector field; and

(S53) computing a depth point for each image point as a function of a distance of each image point from the focus of expansion.

11. The endoscopic surgical method of claim 9, wherein the estimation of the depth field includes:

(S53) computing a depth point for each image point as a function of an amplitude of each vector in the vector field.

12. The endoscopic surgical method of claim 9, wherein the estimation of the depth field includes:

(S53) computing a depth point for an image point as a function of a velocity of each vector in the vector field.

13. The endoscopic surgical method of claim 9, wherein the estimation of the depth field includes:

(S82) computing projection matrices as a function of the vector field.

14. The endoscopic surgical method of claim 13, wherein the estimation of the depth field includes:

(S83) computing a depth point for each image point as a function of a geometric triangulation of projection elements of the projection matrices.

15. The endoscopic surgical method of claim 8, further comprising:

(S43) displaying a depth map (37) representative of the depth field estimation.