Image data processing device and transcranial magnetic stimulation apparatus

• US 9,993,655 B2
• Filed: 03/08/2012
• Issued: 06/12/2018
• Est. Priority Date: 03/09/2011
• Status: Active Grant

• Current Assignee: A School Corporation Kansai University, Osaka University
• Original Assignee: A School Corporation Kansai University, Osaka University
• Inventors: Saitoh, Youichi, Uchida, Kuniyoshi, Hosomi, Koichi, Yasumuro, Yoshihiro, Ogino, Tatsuya
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Rivera-Martinez, Guillermo
• Application Number:
  US14/004,060
• Publication Number:
  US 20130345491A1
• Time in Patent Office: 2,287 Days
• CPC Class Codes: A61B 5/0042 for the brain
• CPC Class Codes: A61B 5/055 involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
• CPC Class Codes: A61B 5/061 Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body A61B1/00158 takes precedence
• CPC Class Codes: A61B 5/1114 Tracking parts of the body
• CPC Class Codes: A61B 5/1128 using image analysis A61B5/1127 takes precedence
• CPC Class Codes: A61B 5/7207 of noise induced by motion artifacts
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: A61N 2/00 Magnetotherapy
• CPC Class Codes: A61N 2/006 for magnetic stimulation of nerve tissue
• CPC Class Codes: A61N 2/02 using magnetic fields produced by coils, including single turn loops or electromagnets A61N2/12 takes precedence
• CPC Class Codes: G06T 11/008 Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10088 Magnetic resonance imaging [MRI]
• CPC Class Codes: G06T 2207/30016 Brain
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/344 involving models
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

An image data processing device including: a storage means configured to store a three-dimensional MRI image of a subject'"'"'s head; a three-dimensional appearance image generation means configured to generate a three-dimensional appearance image of the head; an image generation means configured to perform positional alignment between the three-dimensional MRI image and the three-dimensional appearance image, and to generate a three-dimensional image of the head after the positional alignment; an after-movement image generation means configured to generate, when the subject'"'"'s head has been moved, a three-dimensional subject'"'"'s head image after the movement and the positional alignment; an operational object image generation means configured to generate an operational object image indicating a position of an operational object; and a display means configured to display the three-dimensional image of the subject'"'"'s head after the movement and the operational object image in one image.

Claims

1. A transcranial magnetic stimulation apparatus for applying magnetic stimulation to a specific portion within a patient'"'"'s head using a magnetic-field generation means disposed outside the head, the apparatus comprising:

the magnetic-field generation means which is capable of moving toward a predetermined treatment irradiation position and posture thereof according to an operation of the patient or a patient assistant;

a storage means which stores a three-dimensional MRI image of a patient'"'"'s head that has been previously taken;

a three-dimensional appearance image generation means which generates a three-dimensional appearance image of the patient'"'"'s head in an initial posture;

an image generation means which performs positional alignment between the three-dimensional MRI image and the three-dimensional appearance image, and generates a three-dimensional image of the patient'"'"'s head after the positional alignment;

a means which stores patterns of face characteristics as templates of characteristic regions, the face characteristics being suitable for tracking and being characteristics in the generated three-dimensional appearance image of the patient'"'"'s head in the initial posture;

an after-movement image generation means which generates, when the patient'"'"'s head has been moved, a three-dimensional image of the patient'"'"'s head after the movement;

a means which finds out characteristic points in a face image after the movement using the templates to obtain three-dimensional coordinates of the characteristic regions;

a means which detects changes of the posture from the initial posture using the characteristic regions in the face image in the initial posture and the characteristic regions in the face image after the movement;

a magnetic-field generation means image generation means which generates an image of the magnetic-field generation means that indicates a current position of the magnetic-field generation means operated so as to maintain a positional relation with a specific portion on the three-dimensional MRI image of the patient'"'"'s head;

a display means which displays the three-dimensional image of the patient'"'"'s head after the movement and the image of the magnetic-field generation means in one image; and

a means which stores position and posture information about the predetermined treatment irradiation position and posture of the magnetic-field generating means as target information, in relation to the three-dimensional image of the patient'"'"'s head after the movement,

wherein the predetermined treatment irradiation position and posture of the magnetic-field generating means is indicated as a target on the display means, and wherein the means which stores patterns of face characteristics as templates of characteristic regions is configured to store the patterns of face characteristics as templates of characteristic regions without using any laser projectors.

2. The transcranial magnetic stimulation apparatus according to claim 1, wherein

the image generation means includes:

a selection means which selects points m_i satisfying a predetermined condition from a plurality of points b_j included in the three-dimensional appearance image, the points m_i being selected respectively for N points a_i included in the three-dimensional MRI image;

a parameter determining means which determines, at each of the points m_i selected by the selection means, a rotation matrix R and a parallel translation vector t as parameters for performing rigid-body transformation such that a value of an error function E(R, t) is minimized, the rigid-body transformation converting the points included in the three-dimensional MRI image into the corresponding points included in the three-dimensional appearance image, the error function E(R, t) being configured by a predetermined calculation procedure using the rotation matrix R and the parallel translation vector t; and

a data processing means which performs the rigid-body transformation to each of the points a_i using the rotation matrix R and the parallel translation vector t until the value of the error function E(R, t) becomes equal to or smaller than a predetermined threshold value, and to cause the selection means to select the points m_i respectively, for the points a_i after the conversion, and the parameter determining means to determine the rotation matrix R and the parallel translation vector t.

3. The transcranial magnetic stimulation apparatus according to claim 2, wherein

the selection means selects the points m_i having a smallest Euclidean distance from the plurality of points b_j, the points m_i being selected respectively for the N points a_i.

4. The transcranial magnetic stimulation apparatus according to claim 2, wherein the error function E(R, t) satisfies (Expression 3)

7. The transcranial magnetic stimulation apparatus according to claim 1, wherein the three-dimensional appearance image of the patient'"'"'s head is generated as a three-dimensional image using parallax between images taken from a plurality of viewpoints.

8. The transcranial magnetic stimulation apparatus according to claim 1, wherein the three-dimensional appearance image of the patient'"'"'s head is generated using a time by which one of light and an ultrasonic wave arrives from one viewpoint.

9. The transcranial magnetic stimulation apparatus according to claim 1, wherein a position and a posture of an imaging means of the three-dimensional appearance image generation means is fixed with respect to the operational object, the imaging means being capable of taking an image of the patient'"'"'s head.

10. The transcranial magnetic stimulation apparatus according to claim 2, wherein the three-dimensional appearance image of the patient'"'"'s head is generated as a three-dimensional image using parallax between images taken from a plurality of viewpoints.

12. The transcranial magnetic stimulation apparatus according to claim 2, wherein

the three-dimensional appearance image of the patient'"'"'s head is generated using a time by which one of light and an ultrasonic wave arrives from one viewpoint.

14. The transcranial magnetic stimulation apparatus according to claim 2, wherein

a position and a posture of an imaging means of the three-dimensional appearance image generation means is fixed with respect to the operational object, the imaging means being capable of taking an image of the patient'"'"'s head.

16. The transcranial magnetic stimulation apparatus according to claim 1, wherein

the means which stores patterns of face characteristics as templates of characteristic regions is configured to store the patterns of face characteristics as templates of characteristic regions in response to a user identifying the patterns of face characteristics in the generated three-dimensional appearance image of the patient'"'"'s head in the initial posture.

17. The transcranial magnetic stimulation apparatus according to claim 1, wherein the characteristic regions comprise inner and outer corners of both of the patient'"'"'s eyes.

18. The transcranial magnetic stimulation apparatus according to claim 1, wherein the characteristic regions comprise corners of the patient'"'"'s mouth.

19. The transcranial magnetic stimulation apparatus according to claim 1, wherein the patterns of face characteristics stored as templates of characteristic regions comprise variations in brightness of the patient'"'"'s face.

20. The transcranial magnetic stimulation apparatus according to claim 1, wherein the patterns of face characteristics stored as templates of characteristic regions comprise variations in color of the patient'"'"'s face.

5. A transcranial magnetic stimulation apparatus for applying magnetic stimulation to a specific portion within a subject'"'"'s head using a magnetic-field generation means disposed outside the head, the apparatus comprising:

an image data processing device which tracks a position and an orientation of a subject'"'"'s head, wherein the image data processing device includes;

an image generation means which generates a three-dimensional appearance image of the subject'"'"'s head in an initial posture;

an extraction and storage means which extracts at least one characteristic region from the three-dimensional appearance image to store as a three-dimensional template image;

a means which stores patterns of face characteristics as templates of the characteristic region without using any laser projectors, the face characteristics being suitable for tracking and being characteristics in the generated three-dimensional appearance image of the subject'"'"'s head in the initial posture;

an after-movement image generation means which generates, when the subject'"'"'s head has been moved, a three-dimensional appearance image of the subject'"'"'s head after the movement;

a means which finds out characteristic points in a face image after the movement using the templates to obtain three-dimensional coordinates of the characteristic region;

a means which detects changes of the posture from the initial posture using the characteristic region in the face image in the initial posture and the characteristic regions in the face image after the movement;

a characteristic region determination means which moves the template image over the three-dimensional image of the subject'"'"'s head after the movement, and determines a position at which correlation between the both image data is maximized as a position of the characteristic region after the movement; and

a parameter determining means which determines a rotation matrix R and a parallel translation vector t as parameters for performing rigid-body transformation such that a value of an error function E(R, t) is minimized, the rigid-body transformation converting points included in the characteristic region before the movement into the corresponding points included in the characteristic region after the movement, the error function E(R, t) being configured by a predetermined calculation procedure using the rotation matrix R and the parallel translation vector t.

6. The transcranial magnetic stimulation apparatus according to claim 5, wherein the error function E(R, t) satisfies (Expression 4)

11. The transcranial magnetic stimulation apparatus according to claim 5, wherein the three-dimensional appearance image of the subject'"'"'s head is generated as a three-dimensional image using parallax between images taken from a plurality of viewpoints.

13. The transcranial magnetic stimulation apparatus according to claim 5, wherein

the three-dimensional appearance image of the subject'"'"'s head is generated using a time by which one of light and an ultrasonic wave arrives from one viewpoint.

15. The transcranial magnetic stimulation apparatus according to claim 5, wherein

a position and a posture of an imaging means of the three-dimensional appearance image generation means is fixed with respect to the operational object, the imaging means being capable of taking an image of the subject'"'"'s head.

21. A transcranial magnetic stimulation apparatus for applying magnetic stimulation to a specific portion within a patient'"'"'s head using a magnetic-field generation means disposed outside the head, the apparatus comprising:

the magnetic-field generation means which is capable of moving toward a predetermined treatment irradiation position and posture thereof according to an operation of the patient or a patient assistant;

a storage means which stores a three-dimensional MRI image of a patient'"'"'s head that has been previously taken;

a three-dimensional appearance image generation means which generates a three-dimensional appearance image of the patient'"'"'s head in an initial posture;

an image generation means which performs positional alignment between the three-dimensional MRI image and the three-dimensional appearance image, and generates a three-dimensional image of the patient'"'"'s head after the positional alignment;

a means which stores patterns of face characteristics as templates of characteristic regions, the face characteristics being suitable for tracking and being characteristics in the generated three-dimensional appearance image of the patient'"'"'s head in the initial posture;

an after-movement image generation means which generates, when the patient'"'"'s head has been moved, a three-dimensional image of the patient'"'"'s head after the movement;

a means which finds out characteristic points in a face image after the movement using the templates to obtain three-dimensional coordinates of the characteristic regions;

a means which detects changes of the posture from the initial posture using the characteristic regions in the face image in the initial posture and the characteristic regions in the face image after the movement;

a magnetic-field generation means image generation means which generates an image of the magnetic-field generation means that indicates a current position of the magnetic-field generation means operated so as to maintain a positional relation with a specific portion on the three-dimensional MRI image of the patient'"'"'s head;

a display means which displays the three-dimensional image of the patient'"'"'s head after the movement and the image of the magnetic-field generation means in one image; and

a means which stores position and posture information about the predetermined treatment irradiation position and posture of the magnetic-field generating means as target information, in relation to the three-dimensional image of the patient'"'"'s head after the movement,

wherein the predetermined treatment irradiation position and posture of the magnetic-field generating means is indicated as a target on the display means, and

wherein the means which stores patterns of face characteristics as templates of characteristic regions is configured to store the patterns of face characteristics as templates of characteristic regions without using any reflective markers affixed to the patient.

22. A transcranial magnetic stimulation apparatus for applying magnetic stimulation to a specific portion within a subject'"'"'s head using a magnetic-field generation means disposed outside the head, the apparatus comprising:

an image data processing device which tracks a position and an orientation of a subject'"'"'s head, wherein the image data processing device includes;

an image generation means which generates a three-dimensional appearance image of the subject'"'"'s head in an initial posture;

an extraction and storage means which extracts at least one characteristic region from the three-dimensional appearance image to store as a three-dimensional template image;

a means which stores patterns of face characteristics as templates of the characteristic region without using any reflective markers affixed to the subject, the face characteristics being suitable for tracking and being characteristics in the generated three-dimensional appearance image of the subject'"'"'s head in the initial posture;

an after-movement image generation means which generates, when the subject'"'"'s head has been moved, a three-dimensional appearance image of the subject'"'"'s head after the movement;

a means which finds out characteristic points in a face image after the movement using the templates to obtain three-dimensional coordinates of the characteristic region;

a means which detects changes of the posture from the initial posture using the characteristic region in the face image in the initial posture and the characteristic regions in the face image after the movement;

a characteristic region determination means which moves the template image over the three-dimensional image of the subject'"'"'s head after the movement, and determines a position at which correlation between the both image data is maximized as a position of the characteristic region after the movement; and

a parameter determining means which determines a rotation matrix R and a parallel translation vector t as parameters for performing rigid-body transformation such that a value of an error function E(R, t) is minimized, the rigid-body transformation converting points included in the characteristic region before the movement into the corresponding points included in the characteristic region after the movement, the error function E(R, t) being configured by a predetermined calculation procedure using the rotation matrix R and the parallel translation vector t.