Multiple landmark detection in medical images based on hierarchical feature learning and end-to-end training
First Claim
1. A method of deep learning for multiple landmark detection, the method comprising:
- receiving a plurality of training images;
training a first deep neural network at a first resolution of the training images, the training of the first deep neural network comprising;
learning locations of a first plurality of landmarks; and
learning the spatial relationships between the locations of the first plurality of landmarks;
training a second deep neural network at a second resolution of the training images, the training of the second deep neural network comprising;
learning locations of a second plurality of landmarks; and
learning spatial relationships between the locations of the second plurality of landmarks;
wherein the locations of the second plurality of landmarks are learnt at the second resolution based on the locations of the first plurality of landmarks at the first resolution.
3 Assignments
0 Petitions
Accused Products
Abstract
The present embodiments relate to detecting multiple landmarks in medical images. By way of introduction, the present embodiments described below include apparatuses and methods for detecting landmarks using hierarchical feature learning with end-to-end training. Multiple neural networks are provided with convolutional layers for extracting features from medical images and with a convolutional layer for learning spatial relationships between the extracted features. Each neural network is trained to detect different landmarks using a different resolution of the medical images, and the convolutional layers of each neural network are trained together with end-to-end training to learn appearance and spatial configuration simultaneously. The trained neural networks detect multiple landmarks in a test image iteratively by detecting landmarks at different resolutions, using landmarks detected a lesser resolutions to detect additional landmarks at higher resolutions.
22 Citations
17 Claims
-
1. A method of deep learning for multiple landmark detection, the method comprising:
-
receiving a plurality of training images; training a first deep neural network at a first resolution of the training images, the training of the first deep neural network comprising; learning locations of a first plurality of landmarks; and learning the spatial relationships between the locations of the first plurality of landmarks; training a second deep neural network at a second resolution of the training images, the training of the second deep neural network comprising; learning locations of a second plurality of landmarks; and learning spatial relationships between the locations of the second plurality of landmarks; wherein the locations of the second plurality of landmarks are learnt at the second resolution based on the locations of the first plurality of landmarks at the first resolution. - View Dependent Claims (2)
-
-
3. A method of deep learning for multiple landmark detection, the method comprising:
-
receiving a plurality of training images; training a first deep neural network at a first resolution of the training images, the training of the first deep neural network comprising; learning locations of a first plurality of landmarks; and learning the spatial relationships between the locations of the first plurality of landmarks; training a second deep neural network at a second resolution of the training images, the training of the second deep neural network comprising; learning locations of a second plurality of landmarks; and learning spatial relationships between the locations of the second plurality of landmarks; wherein training the first deep neural network comprises simultaneously learning a plurality of landmark probability maps for the first plurality of landmarks and training the second deep neural network comprises simultaneously learning a plurality of landmark probability maps for the second plurality of landmarks. - View Dependent Claims (4, 5, 6)
-
-
7. A method of deep learning for multiple landmark detection, the method comprising:
-
receiving a plurality of training images; training a first deep neural network at a first resolution of the training images, the training of the first deep neural network comprising; learning locations of a first plurality of landmarks; and learning the spatial relationships between the locations of the first plurality of landmarks; training a second deep neural network at a second resolution of the training images, the training of the second deep neural network comprising; learning locations of a second plurality of landmarks; and learning spatial relationships between the locations of the second plurality of landmarks; wherein the first deep neural network and the second deep neural network each comprise; one or more convolutional layers configured to extract features from the training images; and a convolutional layer configured to map the spatial configuration of the features, wherein the convolutional layers are trained simultaneously. - View Dependent Claims (8)
-
-
9. A system for detecting multiple landmarks in medical image data, the system comprising:
-
a scanner configured to capture medical image data; at least one processor; and at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the system to; receive the medical image data captured by the scanner; detect, using a trained first artificial agent and a trained second artificial agent, multiple landmarks in the medical image data at different resolutions of the medical image data; wherein the first artificial agent and the second artificial agent comprise; convolutional layers of the artificial agents having been trained to extract features from the medical image data; and convolutional layers of the artificial agents having been trained to map spatial relationships between features of the medical image data. - View Dependent Claims (10, 11, 12)
-
-
13. A method for multiple landmark detection, the method comprising:
-
receiving, from a medical scanner, medical image data; identifying, using a first learned deep neural network, a first subset of a plurality of landmarks from the medical image data at a first resolution; identifying, using a second learned deep neural network, a second subset of the plurality of landmarks from the medical image data at a second resolution; and displaying a medical image from the medical image data identifying the identified first subset of landmarks and the identified second subset of landmarks; wherein locations of the first subset of landmarks are used to identify the second subset of landmarks. - View Dependent Claims (14, 15, 16)
-
-
17. A method for multiple landmark detection, the method comprising:
-
receiving, from a medical scanner, medical image data; identifying, using a first learned deep neural network, a first subset of a plurality of landmarks from the medical image data at a first resolution; identifying, using a second learned deep neural network, a second subset of the plurality of landmarks from the medical image data at a second resolution; and displaying a medical image from the medical image data identifying the identified first subset of landmarks and the identified second subset of landmarks; wherein a first number landmarks in the first subset of landmarks and a second number of landmarks in the second subset of landmarks are selected such that identifying the first subset of landmarks and identifying the second subset of landmarks requires substantially the same computational complexity.
-
Specification