INTELLIGENT DISPLAY

• US 20160000517A1
• Filed: 06/29/2015
• Published: 01/07/2016
• Est. Priority Date: 07/02/2014
• Status: Active Grant

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Covidien LP (Medtronic Plc)
• Inventors: KLEIN, EYAL, AVERBUCH, DORIAN, KEHAT, ISRAEL, GREENBURG, BENJAMIN
• Granted Patent: US 11,188,285 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 1/0005 combining images e.g. side-by-side, superimposed or tiled
• CPC Class Codes: A61B 1/2676 Bronchoscopes
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2090/364 Correlation of different images or relation of image positions in respect to the body
• CPC Class Codes: A61B 2090/367 creating a 3D dataset from 2D images using position information
• CPC Class Codes: A61B 2090/374 NMR or MRI
• CPC Class Codes: A61B 2090/3762 using computed tomography systems [CT]
• CPC Class Codes: A61B 2560/0475 Special features of memory means, e.g. removable memory cards
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 5/0013 Medical image data A61B1/00011, A61B6/56, A61B8/56 take precedence
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2211/412 Dynamic
• CPC Class Codes: G06T 2211/428 Real-time
• CPC Class Codes: G09G 2380/08 Biomedical applications

Abstract

A medical image display apparatus for displaying medical images of a lung on a screen includes a network interface receiving positional information of a navigation instrument from a position sensor of the navigation instrument, a video stream from an optical sensor of the navigation instrument, and medical images from an imaging device, a memory storing a plurality of medical images and instructions, a processor executing the instructions, and a display dynamically displaying images on the screen. The instructions, when executed by the processor, cause the medical image display apparatus to determine whether status information indicates a pathway reviewing mode, a target management mode, or a navigation mode. The instructions, when executed by the processor, further cause the display to dynamically select and update images, which are displayed on the screen, among the plurality of medical images based on the positional information of the navigation instrument and status information.

Claims

1. A medical image display apparatus for displaying medical images of a lung on a screen, comprising:

a network interface configured to receive positional information of a navigation instrument from a position sensor of the navigation instrument, a video stream from an optical sensor of the navigation instrument, and medical images from an imaging device;

a memory storing a plurality of medical images and instructions;

a processor configured to execute the instructions; and

a display configured to dynamically display images on the screen,

wherein the instructions, when executed by the processor, cause the medical image display apparatus to determine whether status information indicates a pathway reviewing mode, a target management mode, or a navigation mode; and

wherein the instructions, when executed by the processor, cause the display to dynamically select and update images, which are displayed on the screen, among the plurality of medical images based on the positional information of the navigation instrument and status information.

2. The medical image display apparatus according to claim 1, wherein the navigation instrument is an endoscopic instrument.

3. The medical image display apparatus according to claim 1, wherein the plurality of medical images are selected from the group consisting of sagittal, coronal, or axial images, a three-dimensional (3D) map of the lung, a target, a pathway plan to the target, virtual bronchoscopic video images, live bronchoscopic video images, a maximum intensity projection image, a 3D CT image, a distal tip of the navigation instrument, and any combination thereof.

4. The medical image display apparatus according to claim 3, wherein the sagittal, coronal, or axial images are captured by computed tomography, fluoroscope, computer aided tomography, positron emission tomography, or magnetic resonance imaging.

5. The medical image display apparatus according to claim 3, wherein a displayed image is a composite image, in which a first image of the lung obtained from a first imaging method is overlaid with a second image obtained from another imaging method.

6. The medical image display apparatus according to claim 5, wherein the portion is the target, the first image is captured by computed tomography and the second image is captured by fluoroscope.

7. The medical image display apparatus according to claim 1, wherein the display displays two or more images synchronously corresponding to changes in the positional information.

8. The medical image display apparatus according to claim 3, wherein, when the target is displayed in the 3D map, a sagittal image, a coronal image, and/or an axial image are selected and displayed based on positional information of the target.

9. The medical image display apparatus according to claim 3, wherein changes in the positional information indicate movements of the position sensor of the navigation instrument in the lung.

10. The medical image display apparatus according to claim 3, wherein the axial, coronal, and sagittal images are displayed based on the positional information of the position sensor of the navigation instrument.

11. The medical image display apparatus according to claim 10, wherein each of the axial, coronal, and sagittal images is controlled by a control.

12. The medical image display apparatus according to claim 11, wherein the control includes zooming and panning.

13. The medical image display apparatus according to claim 3, wherein the 3D map is displayed with an orientation indicator.

14. The medical image display apparatus according to claim 13, wherein the orientation indicator shows an orientation of the 3D map.

15. The medical image display apparatus according to claim 3, wherein the display displays the live bronchoscopic video images, when the positional information indicates the position sensor does not pass a threshold position.

16. The medical image display apparatus according to claim 15, wherein the display removes the live bronchoscopic video images and displays the virtual bronchoscopic video images when the positional information indicates the position sensor passes the threshold position.

17. The medical image display apparatus according to claim 3, wherein the display displays a last received image when the status information indicates that no live bronchoscopic video images are received from the navigation instrument in the navigation mode.

18. The medical image display apparatus according to claim 3, wherein the display automatically orients the 3D map to show a current position of the position sensor in the 3D map with sufficient clarity.

19. A medical image display system for displaying medical images on a screen, comprising:

an imaging device configured to captures images of a patient;

a navigation instrument configured to navigate inside of the patient, to transmit positional information obtained by a position sensor and video stream obtained by an optical sensor; and

an apparatus comprising;

a network interface configured to receive the positional information and the video stream from the navigation instrument and the captured images from the imaging device;

a processor configured to execute instructions;

a memory storing a plurality of medical images and the instructions; and

a display configured to display images on the screen,

wherein the instructions, when executed by the processor, cause the apparatus to determine a status of the navigation instrument, which indicates whether the navigation instrument transmits the positional information and the video stream; and

wherein the instructions, when executed by the processor, cause the display to dynamically select and update images, which are displayed on the screen, among a plurality of images based on the positional information and status information of the navigation instrument.

20. A method for dynamically displaying medical images of a lung on a display of a display device, which stores slice images of the lung, three-dimensional (3D) map of the lung, and a pathway plan to a target, the method comprising:

determining whether status information of the display device indicates a pathway reviewing mode, a navigation mode, or a target management mode;

receiving positional information from a navigation instrument, which indicates a position of a position sensor of the navigation instrument navigating the lung; and

displaying medical images based on the determined status information and the positional information,

wherein the displayed medical images include the 3D map and virtual bronchoscopic video images, which are overlaid with the pathway plan and the target, when the determined status information is the pathway reviewing mode,

wherein the displayed medical images include the 3D map, the virtual or live bronchoscopic video images, and slice images, all of which synchronously track the positional of the position sensor, when the determined status information is the navigation mode,

wherein the displayed medical images include three slice images, which are taken from three independent directions, and a maximum intensity projection image displaying the target, when the determined status information is the target management mode, and

wherein when one image is panned or zoomed in the navigation mode, all images and the 3D map are synchronously panned or zoomed correspondingly.