Overlay of anatomical information in a microscope image

• US 10,567,660 B2
• Filed: 03/14/2014
• Issued: 02/18/2020
• Est. Priority Date: 03/14/2014
• Status: Active Grant

• Current Assignee: Brainlab AG
• Original Assignee: Brainlab AG
• Inventors: Elefteriu, Valentin, Mueller, Theodor
• Primary Examiner(s): Navas, Jr., Edemio
• Application Number:
  US15/116,594
• Publication Number:
  US 20160360117A1
• Time in Patent Office: 2,167 Days
• Field of Search: None
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 90/20 Surgical microscopes characterised by non-optical aspects
• CPC Class Codes: A61B 90/25 Supports therefor
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A medical data processing method of determining anatomical structure subset data describing a subset of a graphical representation 9 of an anatomical structure of a patient'"'"'s body to be displayed simultaneously with a medical image of an anatomical body part 1, the method being constituted to be executed by a computer and comprising the following steps: a) acquiring predetermined anatomical structure representation data describing a graphical representation of the anatomical structure and its position in the patient'"'"'s body; b) acquiring anatomical body part image data describing an image of an anatomical body part 1 of the patient imaged by an optical imaging apparatus 2 for display by a display apparatus 3; c) acquiring optical parameter data describing an optical parameter WD serving as a basis for displaying the anatomical body part image data; d) determining, based on the anatomical structure representation data and the anatomical body part image data and the optical parameter data, anatomical structure subset data describing a subset 9 of the anatomical structure representation data, which subset describes graphical representation of a structure subset of the anatomical structure to be displayed simultaneously with the image of the anatomical body part 1.

Claims

1. A medical imaging device, comprising:

an optical imaging apparatus;

a display apparatus operably coupled to the optical image apparatus;

at least one computer that is operably coupled to the optical imaging apparatus and the display apparatus and which comprises at least one processor, the computer being operably coupled to a non-transitory computer-readable data storage medium storing a program which, when executed by the at least one processor, causes the at least one computer to execute medical data processing method of determining anatomical structure subset data describing a subset of a graphical representation of an anatomical structure of a patient'"'"'s body to be displayed simultaneously with a medical image of an anatomical body part, the method comprising executing, by at least one processor of the at least one computer, steps of;

acquiring, at the at least one processor, predetermined anatomical structure representation data describing a graphical representation of the anatomical structure and its position in the patient'"'"'s body;

acquiring, at the at least one processor, anatomical body part image data describing an image of an anatomical body part of the patient imaged by an optical imaging apparatus for display by a display apparatus;

acquiring, at the at least one processor, optical parameter data describing a value of an optical parameter serving as a basis for displaying the anatomical body part image data;

acquiring optical parameter interval data describing a predetermined interval of values for the optical parameter in which the anatomical structure subset data can be determined;

determining, based on the optical parameter data and the optical parameter interval data, whether the value of the optical parameter lies within the interval of values for the optical parameter;

determining, by the at least one processor and on condition that the value of the optical parameter lies within the interval of values for the optical parameter and based on the anatomical structure representation data and the anatomical body part image data and the optical parameter data, anatomical structure subset data describing a real subset of the anatomical structure representation data, which real subset describes a graphical representation of a structure subset of the anatomical structure to be displayed simultaneously with the image of the anatomical body part, the real subset of the anatomical structure representation data comprising less information than the total image information contained in the anatomical structure representation data,

the anatomical structure subset data being determined based on a size of parts of the anatomical structure contained in the structure subset; and

determining, by the at least one processor on condition that the value of the optical parameter lies within the interval of values for the optical parameter, interval indication data describing information to be output to an associated user indicating that the optical parameter lies in the predetermined interval of values for the optical parameter.

2. A medical data processing method of determining anatomical structure subset data describing a subset of a graphical representation of an anatomical structure of a patient'"'"'s body to be displayed simultaneously with a medical image of an anatomical body part, the method comprising executing, by at least one processor of at least one computer, steps of:

acquiring, at the at least one processor, predetermined anatomical structure representation data describing a graphical representation of the anatomical structure and its position in the patient'"'"'s body;

acquiring, at the at least one processor, anatomical body part image data describing an image of an anatomical body part of the patient imaged by an optical imaging apparatus for display by a display apparatus;

acquiring, at the at least one processor, optical parameter data describing a value of an optical parameter serving as a basis for displaying the anatomical body part image data;

acquiring optical parameter interval data describing a predetermined interval of values for the optical parameter in which the anatomical structure subset data can be determined;

determining, based on the optical parameter data and the optical parameter interval data, whether the value of the optical parameter lies within the interval of values for the optical parameter;

determining, by the at least one processor and on condition that the value of the optical parameter lies within the interval of values for the optical parameter and based on the anatomical structure representation data and the anatomical body part image data and the optical parameter data, anatomical structure subset data describing a real subset of the anatomical structure representation data, which real subset describes a graphical representation of a structure subset of the anatomical structure to be displayed simultaneously with the image of the anatomical body part, the real subset of the anatomical structure representation data comprising less information than the total image information contained in the anatomical structure representation data,

the anatomical structure subset data being determined based on a size of parts of the anatomical structure contained in the structure subset; and

determining, by the at least one processor on condition that the value of the optical parameter lies within the interval of values for the optical parameter, interval indication data describing information to be output to an associated user indicating that the optical parameter lies in the predetermined interval of values for the optical parameter.

3. The method according to claim 2, wherein the optical parameter is a specific focus position that is at least one of a real focal length and a working distance at which the image of the anatomical body part is imaged by the optical imaging apparatus.

4. The method according to claim 2, wherein the optical parameter is a specific focus position that is a virtual focal length, at which the image of the anatomical body part is or is to be displayed by the display apparatus.

5. The method according to claim 2, further comprising:

determining, by the at least one processor and based on the anatomical structure subset data and the anatomical body part image data, display data describing a combined image comprising the image of the anatomical body part and an image generated from the subset of the anatomical structure representation data.

6. The method according to claim 5, wherein the combined image includes an overlay of the graphical representation of the subset of the anatomical structure on the image of the anatomical body part.

7. The method according to claim 5 comprising a step of displaying the combined image on the display apparatus.

8. The method according to claim 2, wherein the optical imaging apparatus includes a lens having a variable focus that is a focus lens including a zoom mechanism, and is a digital microscope or a digital camera.

9. The method according to claim 2, wherein the information to be output to a user is visual information that is characterised by a specific colour.

10. The method according to claim 2, further comprising:

determining, by the at least one processor and based on the optical parameter interval data and the optical parameter data, optical parameter variation data describing a direction in which the optical parameter has to be varied to allow determination of the anatomical structure subset data, wherein the interval indication data comprises the optical parameter variation data.

11. The method according to claim 2, wherein

the anatomical structure image data comprises class information describing classes which define structure subsets of the anatomical structure, wherein

each structure subset denotes a part of the anatomical structure which has a predetermined size and is associated with a predetermined value of the optical parameter, and wherein

the anatomical structure subset data is determined, by the at least one processor, based on determining the structure subset which is associated with the actual value of the optical parameter.

12. The method according to claim 11, wherein the size of the part of the anatomical structure contained in a respective one of the classes becomes smaller the smaller or larger the associated value of the optical parameter becomes.

13. A non-transitory computer-readable data storage medium storing a program which, when executed by at least one processor of at least one computer, causes the at least one computer to execute a medical data processing method of determining anatomical structure subset data describing a subset of a graphical representation of an anatomical structure of a patient'"'"'s body to be displayed simultaneously with a medical image of an anatomical body part, the method comprising executing, by the at least one processor of at least one computer, steps of:

acquiring, at the at least one processor, predetermined anatomical structure representation data describing a graphical representation of the anatomical structure and its position in the patient'"'"'s body;

acquiring, at the at least one processor, anatomical body part image data describing an image of an anatomical body part of the patient imaged by an optical imaging apparatus for display by a display apparatus;

acquiring, at the at least one processor, optical parameter data describing a value of an optical parameter serving as a basis for displaying the anatomical body part image data;

acquiring optical parameter interval data describing a predetermined interval of values for the optical parameter in which the anatomical structure subset data can be determined;

determining, based on the optical parameter data and the optical parameter interval data, whether the value of the optical parameter lies within the interval of values for the optical parameter;

determining, by the at least one processor and on condition that the value of the optical parameter lies within the interval of values for the optical parameter and based on the anatomical structure representation data and the anatomical body part image data and the optical parameter data, anatomical structure subset data describing a real subset of the anatomical structure representation data, which real subset describes a graphical representation of a structure subset of the anatomical structure to be displayed simultaneously with the image of the anatomical body part, the real subset of the anatomical structure representation data comprising less information than the total image information contained in the anatomical structure representation data,

the anatomical structure subset data being determined based on a size of parts of the anatomical structure contained in the structure subset; and

determining, by the at least one processor on condition that the value of the optical parameter lies within the interval of values for the optical parameter, interval indication data describing information to be output to an associated user indicating that the optical parameter lies in the predetermined interval of values for the optical parameter.

14. A computer operably coupled to the non-transitory computer-readable data storage medium according to claim 13.