System and method for increasing the accuracy of a medical imaging device

• US 10,335,108 B2
• Filed: 03/04/2016
• Issued: 07/02/2019
• Est. Priority Date: 03/04/2015
• Status: Active Grant

• Current Assignee: Newton2 APS
• Original Assignee: 3shape A/S
• Inventors: Boll, Morten, Sorensen, Thomas Sangild, Holme, Jens Peter, Brandt, Sami, Ojelund, Henrik
• Primary Examiner(s): Azarian, Seyed H
• Application Number:
  US15/555,502
• Publication Number:
  US 20180042570A1
• Time in Patent Office: 1,215 Days
• Field of Search: 382100, 382103, 382106-107, 382128-134, 382154-155, 382162, 382168, 382173, 382181, 382189, 382199, 382219, 382224, 382232, 382254, 382274, 382276, 382286-291, 382305, 382312, 378 4, 378 21, 378162, 378 63, 600 1, 600424, 433 29
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/0492 using markers or indicia for aiding patient positioning
• CPC Class Codes: A61B 6/4085 Cone-beams
• CPC Class Codes: A61B 6/501 for diagnosis of the head, e.g. neuroimaging or craniography
• CPC Class Codes: A61B 6/5229 combining image data of a patient, e.g. combining a functional image with an anatomical image
• CPC Class Codes: A61B 6/5264 due to motion
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A method for improving the accuracy of a digital medical model of a part of a patient, the method includes obtaining a set of at least 2 medical images of the patient, where an element comprising a predefined geometry and/or predefined information was attached to the patient during the recording of the medical images; obtaining at least 2 tracking images taken with at least one camera having a known positional relationship relative to the medical imaging device, said tracking images depicting at least part of the element; determining any movement of the element between acquisition of the at least 2 tracking images; and generating the digital medical model from the acquired medical images, wherein the determined movement of the element is used to compensate for any movement of the patient between the acquisition of the medical images.

Claims

1. A method for improving the accuracy of a digital medical model of a part of a patient, the medical model comprising at least 2 medical images recorded with a medical imaging device, the method comprising:

obtaining a set of at least 2 medical images of the patient, where an element comprising a predefined geometry or predefined information was attached to the patient during the recording of the medical images;

obtaining at least 2 tracking images taken with at least one camera having a known positional relationship relative to the medical imaging device, said tracking images depicting at least part of the element;

determining any movement of the element between acquisition of the at least 2 tracking images; and

generating the digital medical model from the acquired medical images, wherein the determined movement of the element is used to compensate for any movement of the patient between the acquisition of the medical images;

wherein compensating for any movement of the patient between the acquisition of the medical images comprises;

associating a time stamp with each of the medical images, and each of the tracking images;

determining the position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition; and

generating the digital medical model from the acquired medical images, wherein the generating of the digital medical model comprises iteratively adjusting the digital medical model to account for the movement of the element during medical image acquisition.

2. The method according to claim 1, wherein the predefined information of the element comprises at least one fiducial marker.

3. The method according to claim 2, wherein the predefined information of the element comprises a plurality of fiducial markers in a predefined pattern, size, shape or colour.

4. The method according to claim 1, wherein the predefined information of the element comprises the geometric shape of the element.

5. The method according to claim 1, wherein compensating for any movement of the patient between the acquisition of the medical images comprises:

associating a time stamp with each of the medical images, and each of the tracking images;

determining the position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition; and

adjusting the position of each pixel or voxel of the acquired medical image with an amount corresponding to the movement of the element.

6. The method according to claim 1, wherein the coordinate system used in determining the position and orientation of the element can be fixed arbitrarily.

7. The method according to claim 1, wherein the tracking images taken and the medical images are time stamped using the same clock.

8. The method according to claim 1, wherein recognizing the fiducial markers comprises performing principal component analysis on the tracking images in order to segment the fiducial markers.

9. The method according to claim 1, wherein there are three cameras for recording the movement of the element, such as three cameras placed at the points of an equilateral triangle.

10. The method according to claim 9, wherein:

the tracking images from the three cameras are acquired simultaneously such that at each time t, there are three images taken of the element;

the fiducial markers are recognized in each image;

the pattern of the fiducial markers is recognized in each tracking image to achieve a best fit to the known predefined pattern of the fiducial markers on the element from each tracking image;

the position and orientation of the element in each of the three tracking images each time t is determined; and

a weighted average of the position and orientation of the element from the three images is used to determine the best fit for the position and orientation of the element at each time t.

11. The method according to claim 9, wherein:

the camera position and rotation of each camera is calibrated or determined;

the intrinsic parameters such as the focal length, skew, principal point and lens distortion are calibrated or determined for each camera;

the tracking images from the three cameras are acquired simultaneously such that at each time t, there are three images taken of the element;

the fiducial markers are recognized in each tracking image and the position of each fiducial marker is determined directly in the camera co-ordinate frame; and

the position or orientation of the element from the three images is determined using a cost function to minimise the difference in the determined position of the fiducial markers in each of the tracking images.

12. The method according to claim 1, wherein the digital medical model is generated in real time.

13. The method according to claim 1, wherein the medical imaging device is a cone beam computed tomography device.

14. The method according to claim 1, where an element comprising a predefined geometry and predefined information was attached to the patient during the recording of the medical images.

15. A method, for improving the accuracy of a digital medical model of a part of a patient, the medical model comprising at least 2 medical images recorded with a medical imaging device, the method comprising:

obtaining a set of at least 2 medical images of the patient, where an element comprising a predefined geometry or predefined information was attached to the patient during the recording of the medical images;

obtaining at least 2 tracking images taken with at least one camera having a known positional relationship relative to the medical imaging device, said tracking images depicting at least part of the element;

determining any movement of the element between acquisition of the at least 2 tracking images;

generating the digital medical model from the acquired medical images, wherein the determined movement of the element is used to compensate for any movement of the patient between the acquisition of the medical images;

wherein compensating for any movement of the patient between the acquisition of the medical images comprises;

associating a time stamp with each of the medical images, and each of the tracking images;

determining the position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition; and

generating the digital medical model from the acquired medical images, wherein the generating of the digital medical model comprises iteratively adjusting the digital medical model to account for the movement of the element during medical image acquisition;

wherein determining the position and orientation of the element at each time stamp comprises;

recognizing a plurality of the individual fiducial markers in each tracking image;

obtaining a digital representation in a database of the known predefined pattern or shape of the fiducial markers; and

recognizing the pattern of the fiducial markers in each image to achieve a best fit to the known predefined pattern of the fiducial markers on the element from each tracking image.

16. The method according to claim 15, where an element comprising a predefined geometry and predefined information was attached to the patient during the recording of the medical images.

17. A method, for improving the accuracy of a digital medical model of a part of a patient, the medical model comprising at least 2 medical images recorded with a medical imaging device, the method comprising:

obtaining a set of at least 2 medical images of the patient, where an element comprising a predefined geometry or predefined information was attached to the patient during the recording of the medical images;

obtaining at least 2 tracking images taken with at least one camera having a known positional relationship relative to the medical imaging device, said tracking images depicting at least part of the element;

determining any movement of the element between acquisition of the at least 2 tracking images;

generating the digital medical model from the acquired medical images, wherein the determined movement of the element is used to compensate for any movement of the patient between the acquisition of the medical images;

wherein compensating for any movement of the patient between the acquisition of the medical images comprises;

associating a time stamp with each of the medical images, and each of the tracking images;

determining the position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition; and

generating the digital medical model from the acquired medical images, wherein the generating of the digital medical model comprises iteratively adjusting the digital medical model to account for the movement of the element during medical image acquisition;

wherein determining the position and orientation of the element at each time stamp comprises;

recognizing a plurality of the individual fiducial markers in each tracking image;

using a classification of the indices of the fiducial markers; and

matching the known pattern of the fiducial markers on the element to the pattern of the fiducial markers on the tracking image using the classification of the indices of the fiducial markers.

18. The method according to claim 17, where an element comprising a predefined geometry and predefined information was attached to the patient during the recording of the medical images.

19. A method for improving the accuracy of a digital medical model of a patient, the digital medical model being computed based on images recorded with a medical imaging device, the method comprising:

obtaining a first tracking image taken with at least one camera having a known positional relationship relative to the medical imaging device, said tracking image depicting at least part of an element comprising a predefined geometry or predefined information;

obtaining a first medical image of the patient, where an element comprising a predefined geometry or predefined information was attached to the patient during the recording of the medical image;

obtaining at second tracking image taken with the at least one camera having a, said tracking image depicting at least part of the element;

determining any movement of the element between acquisition of the 2 tracking images;

generating the digital medical model wherein the determined movement of the element is used to compensate for any movement of the patient relative to the medical imaging device during the acquisition of the medical image;

wherein compensating for any movement of the patient between the acquisition of the medical images comprises;

associating a time stamp with each of the medical images, and each of the tracking images;

determining the position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition; and

generating the digital medical model from the acquired medical images, wherein the generating of the digital medical model comprises iteratively adjusting the digital medical model to account for the movement of the element during medical image acquisition.

20. The method according to claim 19, wherein the predefined information of the element comprises at least one fiducial marker.

21. The method according to claim 20, wherein the predefined information of the element comprises a plurality of fiducial markers in a predefined pattern, size, shape or colour.

22. The method according to claim 19, wherein the predefined information of the element comprises the geometric shape of the element.

23. The method according to claim 19, wherein compensating for any movement of the patient between the acquisition of the medical images comprises:

associating a time stamp with each of the medical images, and each of the tracking images;

determining the position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition; and

adjusting the position of each pixel or voxel of the acquired medical image with an amount corresponding to the movement of the element.

24. The method according to claim 23, wherein determining the position and orientation of the element at each time stamp comprises:

recognizing a plurality of the individual fiducial markers in each tracking image;

obtaining a digital representation in a database of the known predefined pattern or shape of the fiducial markers;

recognizing the pattern of the fiducial markers in each image to achieve a best fit to the known predefined pattern of the fiducial markers on the element from each tracking image.

25. The method according to claim 23, wherein determining the position and orientation of the element at each time stamp comprises:

recognizing a plurality of the individual fiducial markers in each tracking image;

using a classification of the indices of the fiducial markers; and

matching the known pattern of the fiducial markers on the element to the pattern of the fiducial markers on the tracking image using the classification of the indices of the fiducial markers.

26. The method according to claim 23, wherein recognizing the fiducial markers comprises performing principal component analysis on the tracking images in order to segment the fiducial markers.

27. The method according to claim 23, wherein there are three cameras for recording the movement of the element, such as three cameras placed at the points of an equilateral triangle.

28. The method according to claim 27, wherein:

the tracking images from the three cameras are acquired simultaneously such that at each time t, there are three images taken of the element;

the fiducial markers are recognized in each image;

the pattern of the fiducial markers is recognized in each tracking image to achieve a best fit to the known predefined pattern of the fiducial markers on the element from each tracking image;

the position and orientation of the element in each of the three tracking images each time t is determined; and

a weighted average of the position and orientation of the element from the three images is used to determine the best fit for the position and orientation of the element at each time t.

29. The method according to claim 27, wherein:

the camera position and rotation of each camera is calibrated or determined;

the intrinsic parameters such as the focal length, skew, principal point and lens distortion are calibrated or determined for each camera;

the tracking images from the three cameras are acquired simultaneously such that at each time t, there are three images taken of the element;

the fiducial markers are recognized in each tracking image and the position of each fiducial marker is determined directly in the camera co-ordinate frame;

the position or orientation of the element from the three images is determined using a cost function to minimise the difference in the determined position of the fiducial markers in each of the tracking images.

30. The method according to claim 19, wherein the coordinate system used in determining the position and orientation of the element can be fixed arbitrarily.

31. The method according to claim 19, wherein the tracking images taken and the medical images are time stamped using the same clock.

32. The method according to claim 19, wherein the medical imaging device is a cone beam computed tomography device.

33. The method according to claim 19, wherein said tracking image depicting at least part of an element comprising a predefined geometry and predefined information;

and

where an element comprising a predefined geometry and predefined information was attached to the patient during the recording of the medical image.

34. A medical imaging system comprising:

a radiation source emitting a radiation beam;

a radiation sensor for detecting incident radiation from the radiation beam on a sensor area;

an element attachable to a patient, the element comprising a number of fiducial markers in a predefined pattern, size, shape or colour;

at least one camera configured to take at least 2 tracking images of the element during the acquisition of medical images taken using the radiation source and radiation sensor;

computer for determining a movement based on the at least 2 tracking images, and for adjusting the medical images acquired using the radiation sensor to compensate for the movement;

wherein compensating for any movement of the patient between the acquisition of the medical images comprises;

associating a time stamp with each of the medical images, and each of the tracking images;

determining the position and orientation of the element at each time stamp and determining therefrom the movement of the element during medical image acquisition; and

generating the digital medical model from the acquired medical images, wherein the generating of the digital medical model comprises iteratively adjusting the digital medical model to account for the movement of the element during medical image acquisition.

35. The medical imaging system according to claim 34, wherein the element comprises a number of fiducial markers in a predefined pattern, size, shape and colour.