Body landmark detection based on depth images

• US 10,506,984 B2
• Filed: 01/05/2018
• Issued: 12/17/2019
• Est. Priority Date: 01/05/2018
• Status: Active Grant

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Yu, Lequan, Ma, Kai, Singh, Vivek, Chen, Terrence
• Primary Examiner(s): Couso, Jose L
• Application Number:
  US15/863,138
• Publication Number:
  US 20190209098A1
• Time in Patent Office: 711 Days
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/75 involving models

Abstract

A system and method includes acquisition of a plurality of sets of body surface data, first data indicating locations of a first one or more body landmarks for each of the plurality of sets of body surface data, and second data indicating locations of a second one or more body landmarks for each of the plurality of sets of body surface data, training, using the plurality of sets of body surface data and data indicating locations of the first one or more body landmarks for each of the plurality of sets of body surface data, of a first reinforcement learning network to identify the first one or more body landmarks based on body surface data, and training, using the plurality of sets of body surface data and data indicating locations of the second one or more body landmarks for each of the plurality of sets of body surface data, of a second reinforcement learning network to identify the second one or more body landmarks based on body surface data.

Claims

1. A system comprising:

a memory configured to store a plurality of sets of body surface data and, for each set of body surface data, data of a plurality of landmark traces, where each of the plurality of landmark traces is associated with one or more body landmark locations; and

a processor configured to execute processor-executable program code to;

train, using the plurality of sets of body surface data and data of a first landmark trace of each of the plurality of sets of body surface data, a first reinforcement learning network to identify a first one or more body landmark locations associated with the first landmark trace, where a result is a trained first reinforcement learning network; and

train, using the plurality of sets of body surface data and data of a second landmark trace of each of the plurality of sets of body surface data, a second reinforcement learning network to identify a second one or more body landmark locations associated with the second landmark trace, where a result is a trained second reinforcement learning network.

2. A system according to claim 1, wherein the training of the first reinforcement learning network and the second reinforcement learning network is contemporaneous.

3. A system according to claim 1, wherein the processor is further configured to execute processor-executable program code to:

input a set of surface data associated with a body to the trained first reinforcement learning network;

input the set of surface data associated with the body to the trained second reinforcement learning network;

determine, using the trained first reinforcement learning network, a first one or more body landmark locations of the body based on the set of surface data; and

determine, using the trained second reinforcement learning network, a second one or more body landmark locations of the body based on the set of surface data.

4. A system according to claim 3, further comprising:

an imaging system to acquire the set of surface data associated with the body.

5. A system according to claim 1, wherein the processor is further configured to execute processor-executable program code to:

train, using the plurality of sets of body surface data and data of a third landmark trace of each of the plurality of sets of body surface data, a third reinforcement learning network to identify a third one or more body landmark locations associated with the third landmark trace, where a result is a trained third reinforcement learning network.

6. A system according to claim 5, wherein the training of the first reinforcement learning network, the second reinforcement learning network and the third reinforcement learning network is contemporaneous, and wherein the processor is further configured to execute processor-executable program code to:

input a set of surface data associated with a body to the trained first reinforcement learning network;

input the set of surface data associated with the body to the trained second reinforcement learning network;

input the set of surface data associated with the body to the trained third reinforcement learning network;

determine, using the trained first reinforcement learning network, a first one or more body landmark locations of the body based on the set of surface data;

determine, using the trained second reinforcement learning network, a second one or more body landmark locations of the body based on the set of surface data; and

determine, using the trained third reinforcement learning network, a third one or more body landmark locations of the body based on the set of surface data.

7. A computer-implemented method, comprising:

acquiring a plurality of sets of body surface data, first data indicating locations of a first one or more body landmarks for each of the plurality of sets of body surface data, and second data indicating locations of a second one or more body landmarks for each of the plurality of sets of body surface data;

training, using the plurality of sets of body surface data and data indicating locations of the first one or more body landmarks for each of the plurality of sets of body surface data, a first reinforcement learning network to identify the first one or more body landmarks based on body surface data, where a result is a trained first reinforcement learning network; and

training, using the plurality of sets of body surface data and data indicating locations of the second one or more body landmarks for each of the plurality of sets of body surface data, a second reinforcement learning network to identify the second one or more body landmarks based on body surface data, where a result is a trained second reinforcement learning network.

8. A method according to claim 7, wherein the training of the first reinforcement learning network and the second reinforcement learning network is contemporaneous.

9. A method according to claim 7, further comprising:

inputting a set of body surface data associated with a body to the trained first reinforcement learning network;

inputting the set of surface data associated with the body to the trained second reinforcement learning network;

determining, using the trained first reinforcement learning network, locations of a first one or more body landmarks of the body based on the set of surface data; and

determining, using the trained second reinforcement learning network, locations of a second one or more body landmarks of the body based on the set of surface data.

10. A method according to claim 9, further comprising:

acquiring the set of surface data associated with the body.

11. A method according to claim 7, further comprising:

training, using the plurality of sets of body surface data and data indicating locations of a third one or more body landmarks for each of the plurality of sets of body surface data, a third reinforcement learning network to identify the third one or more body landmarks based on body surface data.

12. A method according to claim 11, wherein the training of the first reinforcement learning network, the second reinforcement learning network and the third reinforcement learning network is contemporaneous, and further comprising:

inputting a set of surface data associated with a body to the trained first reinforcement learning network;

inputting the set of surface data associated with the body to the trained second reinforcement learning network;

inputting the set of surface data associated with the body to the trained third reinforcement learning network;

determining, using the trained first reinforcement learning network, locations of a first one or more body landmarks of the body based on the set of surface data;

determining, using the trained second reinforcement learning network, locations of a second one or more body landmarks of the body based on the set of surface data; and

determining, using the trained third reinforcement learning network, locations of a third one or more body landmarks of the body based on the set of surface data.

13. A non-transitory computer-readable medium storing processor-executable process steps, the process steps executable by a processor to cause a system to:

acquire a plurality of sets of body surface data, first data indicating locations of a first one or more body landmarks for each of the plurality of sets of body surface data, and second data indicating locations of a second one or more body landmarks for each of the plurality of sets of body surface data;

train, using the plurality of sets of body surface data and data indicating locations of the first one or more body landmarks for each of the plurality of sets of body surface data, a first reinforcement learning network to identify the first one or more body landmarks based on body surface data, where a result is a trained first reinforcement learning network; and

train, using the plurality of sets of body surface data and data indicating locations of the second one or more body landmarks for each of the plurality of sets of body surface data, a second reinforcement learning network to identify the second one or more body landmarks based on body surface data, where a result is a trained second reinforcement learning network.

14. A medium according to claim 13, wherein the training of the first reinforcement learning network and the second reinforcement learning network is contemporaneous.

15. A medium according to claim 13, the process steps executable by the processor to cause a system to:

input a set of body surface data associated with a body to the trained first reinforcement learning network;

input the set of surface data associated with the body to the trained second reinforcement learning network;

determine, using the trained first reinforcement learning network, locations of a first one or more body landmarks of the body based on the set of surface data; and

determine, using the trained second reinforcement learning network, locations of a second one or more body landmarks of the body based on the set of surface data.

16. A medium according to claim 15, the process steps executable by the processor to cause a system to:

acquire the set of surface data associated with the body.

17. A medium according to claim 13, the process steps executable by the processor to cause a system to:

train, using the plurality of sets of body surface data and data indicating locations of a third one or more body landmarks for each of the plurality of sets of body surface data, a third reinforcement learning network to identify the third one or more body landmarks based on body surface data.

18. A medium according to claim 17, wherein the training of the first reinforcement learning network, the second reinforcement learning network and the third reinforcement learning network is contemporaneous, and wherein the process steps executable by the processor to cause a system to:

input a set of surface data associated with a body to the trained first reinforcement learning network;

input the set of surface data associated with the body to the trained second reinforcement learning network;

input the set of surface data associated with the body to the trained third reinforcement learning network;

determine, using the trained first reinforcement learning network, locations of a first one or more body landmarks of the body based on the set of surface data;

determine, using the trained second reinforcement learning network, locations of a second one or more body landmarks of the body based on the set of surface data; and

determine, using the trained third reinforcement learning network, locations of a third one or more body landmarks of the body based on the set of surface data.