MACHINE LEARNING APPROACH TO REAL-TIME PATIENT MOTION MONITORING

• US 20200129780A1
• Filed: 10/25/2018
• Published: 04/30/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: Elekta Incorporated (Elekta AB)
• Original Assignee: Elekta Incorporated (Elekta AB)
• Inventors: Lachaine, Martin Emile, Beriault, Silvain
• Granted Patent: US 11,083,913 B2
• Time in Patent Office: Days
• CPC Class Codes: A61N 2005/1034 Monte Carlo type methods; particle tracking
• CPC Class Codes: A61N 2005/1052 using positron emission tomography [PET] single photon emission computer tomography [SPECT] imaging
• CPC Class Codes: A61N 2005/1054 using a portal imaging system
• CPC Class Codes: A61N 2005/1055 using magnetic resonance imaging [MRI]
• CPC Class Codes: A61N 2005/1058 using ultrasound imaging
• CPC Class Codes: A61N 2005/1061 using an x-ray imaging system having a separate imaging source
• CPC Class Codes: A61N 5/1037 taking into account the movement of the target, e.g. 4D-image based planning
• CPC Class Codes: A61N 5/1039 using functional images, e.g. PET or MRI
• CPC Class Codes: A61N 5/107 in real time, i.e. during treatment
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 50/50 for simulation or modelling of medical disorders
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Systems and techniques may be used to estimate a patient state during a radiotherapy treatment. For example, a method may include generating a dictionary of expanded potential patient measurements and corresponding potential patient states using a preliminary motion model. The method may include training, using a machine learning technique, a correspondence motion model relating an input patient measurement to an output patient state using the dictionary. The method may include estimating, using a processor, the patient state corresponding to an input image using the correspondence motion model.

Claims

1. A method for estimating a real-time patient state during a radiotherapy treatment, the method comprising:

identifying, using a processor, a preliminary motion model of a patient under motion;

generating a dictionary of expanded potential patient measurements and corresponding potential patient states using the preliminary motion model;

training, using a machine learning technique, a correspondence motion model relating an input patient measurement to an output patient state using the dictionary; and

estimating, using the processor, the patient state corresponding to a patient measurement of the patient using the correspondence motion model.

2. The method of claim 1, wherein the corresponding patient states include a 3D patient image.

3. The method of claim 2, wherein the corresponding potential patient states include deformations of the 3D patient image.

4. The method of claim 3, wherein the deformations include deformation vector fields (DVFs) calculated using a deformable registration algorithm.

5. The method of claim 4, wherein the DVFs are 3D DVFs applied to the correspondence motion model to generate the patient state using deformation.

6. The method of claim 3, wherein the deformations include parameterizations of 3D DVFs and the preliminary motion model includes a reference image.

7. The method of claim 1, wherein the patient measurements include a 2D MRI slice, MRI k-space data, a 1D MRI navigator, a 2D MRI projection, x-ray 2D projection data, PET data, or a 2D ultrasound slice.

8. The method of claim 1, wherein the patient data includes a 4D image.

9. The method of claim 8, wherein the 4D image is a 4D CT, a 4D CBCT, a 4D MRI, a 4D PET, or a 4D ultrasound image.

10. The method of claim 1, wherein the correspondence motion model includes a deformation vector field (DVF) as a function of one or more parameters, the one or more parameters determined by reducing dimensionality of a preliminary DVF calculated between two or more phases of a 4D image and a reference phase.

11. The method of claim 1, wherein the expanded potential patient measurements include a 2D projection image and are generated by using at least one of:

extracting a 2D slice from a 3D image, ray-tracing through a 3D image to generate a 2D projection image, simulating x-ray interactions with a 3D image using a Monte Carlo technique, using a collapsed cone convolution technique, using a superposition and convolution technique, using a generative adversarial network, a convolutional neural network, or a recurrent neural network.

12. The method of claim 1, wherein the correspondence motion model is generated using a random forest regression, a linear regression, a polynomial regression, a regression tree, a kernel density estimation, a support vector regression algorithm, a convolutional neural network, or a recurrent neural network.

13. The method of claim 1, wherein estimating the patient state corresponding to the patient measurement includes receiving the patient measurement as an input to the correspondence motion model, the input including a real-time stream of 2D images.

14. The method of claim 13, wherein the real-time stream of 2D images includes stereoscopic kV images or pairs of 2D MR slice images.

15. The method of claim 1, further comprising outputting the patient state as two or more MR-like 3D images showing tissue contrast.

16. The method of claim 1, wherein the patient state includes non-imaging information.

17. The method of claim 1, further comprising generating the preliminary motion model based on a 4D dataset acquired before the radiotherapy treatment.

18. The method of claim 1, further comprising generating the expanded potential patient measurements by calculating a 2D deformation vector field (DVF) on a 2D input image.

19. The method of claim 1, wherein generating the expanded potential patient measurements includes performing a principal component analysis (PCA) analysis of the 2D input image.

20. The method of claim 1, wherein generating the expanded potential patient measurements includes registering the 2D input image to a reference 2D image, and using a deformable image registration technique to calculate the 2D DVF.

21. The method of claim 1, wherein generating the expanded potential patient measurements includes using a convolutional neural network (CNN) to estimate a 2D optical flow between the 2D input image and a 2D reference image to calculate the 2D DVF.

22. A method for generating real-time target localization data, the method comprising:

generating a dictionary of expanded potential patient measurements and corresponding potential patient states using a preliminary motion model;

training, using a machine learning technique, a correspondence motion model relating an input patient measurement to an output patient state using the dictionary;

receiving a real-time stream of 2D images from an image acquisition device;

estimating, using the processor, the patient state corresponding to an image of the real-time stream of 2D images using the correspondence motion model;

locating a radiation therapy target within a patient using the patient state; and

outputting the location of the radiation therapy target on a display device.

23. The method of claim 22, wherein the real-time stream of 2D images includes stereoscopic kV images or pairs of 2D MR slice images.

24. The method of claim 22, wherein the expanded potential patient measurements include deformations of a 3D patient image, and wherein the deformations include deformation vector fields (DVFs) calculated using a deformable registration algorithm.

25. The method of claim 22, wherein the expanded potential patient measurements are generated from a 4D image, the 4D image including a 4D CT, a 4D CBCT, a 4D MRI, a 4D PET, or a 4D ultrasound image.

26. The method of claim 22, wherein the correspondence motion model includes a deformation vector field (DVF) as a function of one or more parameters, the one or more parameters determined by reducing dimensionality of a preliminary DVF calculated between two or more phases of a 4D image and a reference phase.

27. The method of claim 22, wherein the expanded potential patient measurements include a 2D projection image and are generated by using at least one of:

extracting a 2D slice from a 3D image, ray-tracing through a 3D image to generate a 2D projection image, simulating x-ray interactions with a 3D image using a Monte Carlo technique, using a collapsed cone convolution technique, using a superposition and convolution technique, using a generative adversarial network, a convolutional neural network, or a recurrent neural network.

28. The method of claim 22, wherein the correspondence motion model is generated using a random forest regression, a linear regression, a polynomial regression, a regression tree, a kernel density estimation, a support vector regression algorithm, a convolutional neural network, or a recurrent neural network.

29. The method of claim 22, further comprising outputting the patient state as two or more MR-like 3D images showing tissue contrast.

30. The method of claim 22, wherein generating the expanded potential patient measurements includes performing a principal component analysis (PCA) analysis of the 2D input image.

31. The method of claim 22, wherein generating the expanded potential patient measurements includes registering the 2D input image to a reference 2D image, and using a deformable image registration technique to calculate the 2D DVF.

32. The method of claim 22, wherein generating the expanded potential patient measurements includes using a convolutional neural network (CNN) to estimate a 2D optical flow between the 2D input image and a 2D reference image to calculate the 2D DVF.

33. A method for real-time tracking of a target, the method comprising:

generating a dictionary of expanded potential patient measurements and corresponding potential patient states using a preliminary motion model;

training, using a machine learning technique, a correspondence motion model relating an input patient measurement to an output patient state using the dictionary;

receiving a real-time stream of 2D images from an image acquisition device;

estimating, using the processor, patient states corresponding to images in the real-time stream of 2D images using the correspondence motion model;

tracking a radiation therapy target of a patient in real-time using the patient states; and

outputting tracking information for the radiation therapy target for display on a display device.

34. The method of claim 33, wherein the real-time stream of 2D images includes stereoscopic kV images or pairs of 2D MR slice images.

35. The method of claim 33, further comprising generating the expanded potential patient measurements by calculating a 2D deformation vector field (DVF) on a 2D input image.

36. The method of claim 33, wherein generating the expanded potential patient measurements includes performing a principal component analysis (PCA) analysis of the 2D input image.

37. The method of claim 33, wherein generating the expanded potential patient measurements includes registering the 2D input image to a reference 2D image, and using a deformable image registration technique to calculate the 2D DVF.

38. The method of claim 33, wherein generating the expanded potential patient measurements includes using a convolutional neural network (CNN) to estimate a 2D optical flow between the 2D input image and a 2D reference image to calculate the 2D DVF.

39. A method for generating real-time target localization data, the method comprising:

generating, using a processor, a 3D deformation vector field (DVF) parameterized by two or more scalars from a 4D image by;

calculating DVFs between each phase of the 4D image and a reference image; and

performing a principal component analysis (PCA) analysis on the DVFs;

generating potential 3D images by;

randomly sampling the two or more scalars;

generating new 3D DVFs from the randomly sampled two or more scalars; and

deforming the reference image to generate corresponding patient states;

associating the corresponding patient states with respective potential patient measurements corresponding to the randomly sampled two or more scalars;

calculating corresponding 2D DVFs between the respective potential patient measurements and a portion of the reference image;

performing a PCA on the corresponding 2D DVFs resulting in expanded potential patient measurements;

training, using a machine learning technique, a correspondence motion model relating the expanded potential patient measurements to the corresponding patient states using a dictionary;

receiving a 2D image of a patient during radiotherapy treatment;

in real-time, calculating a measurement by;

calculating a 2D DVF between the 2D image and a portion of the reference image; and

performing a PCA on the 2D DVF;

inputting the measurement to the correspondence motion model to generate PCA components of a reconstructed 3D DVF;

generating the reconstructed 3D DVF from the PCA components; and

deforming the reference image with the reconstructed 3D DVF to generate a current real-time 3D patient image.

40. The method of claim 39, further comprising outputting a current real-time patient state comprising the current real-time 3D patient state or the reference image and the reconstructed 3D DVF.

41. The method of claim 39, wherein the respective potential patient measurements include a 2D image extracted from a 3D image or a 2D projection through a 3D image.

42. The method of claim 39, wherein the 4D image is a 4D MR image or a 4D CBCT image.