IMAGE-PROCESSING APPARATUS, IMAGE-PROCESSING METHOD, AND PROGRAM

• US 20160270757A1
• Filed: 11/11/2013
• Published: 09/22/2016
• Est. Priority Date: 11/15/2012
• Status: Abandoned Application

• Current Assignee: Konica Minolta Inc.
• Original Assignee: Konica Minolta Inc.
• Inventors: TOMA, Tadamasa, OHMIYA, Jun, TOJI, Bumpei
• Application Number:
  US14/442,281
• Publication Number:
  US 20160270757A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 8/0841 for locating instruments
• CPC Class Codes: A61B 8/0891 for diagnosis of blood vessels
• CPC Class Codes: A61B 8/4483 characterised by features of the ultrasound transducer
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/466 adapted to display 3D data 3D imaging and stereoscopic displays in devices for short-range imaging with acoustic waves G01S15/8993 and G01S7/52068, respectively; 3D image rendering G06T15/00; manipulating 3D models for computer graphics G06T19/00
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/5207 involving processing of raw data to produce diagnostic data, e.g. for generating an image reconstruction G06T11/00
• CPC Class Codes: A61B 8/5223 for extracting a diagnostic or physiological parameter from medical diagnostic data for algorithms to analyse biomedical images G06T7/0012
• CPC Class Codes: A61B 8/5246 combining images from the same or different imaging techniques, e.g. color Doppler and B-mode
• CPC Class Codes: G16H 50/30 for calculating health indices; for individual health risk assessment

Abstract

An image processing apparatus capable of displaying an image for providing guidance in moving an instrument to a target part in a subject to a user with high visual perceptibility. The image processing apparatus may be an ultrasound diagnostic apparatus including: a three-dimensional image analyzer determining target position indicating a three-dimensional position of the target part based on a three-dimensional image including the target part; a position information acquirer acquiring instrument position indicating a three-dimensional position of the instrument; a display state determiner selecting one display state from at least two display states, based on a positional relationship between the target part and the instrument; an assist image generator generating an assist image for the selected display state by using the target position and the instrument position; and a display controller performing control for outputting the assist image generated by the assist image generation unit to a display device.

Claims

1. An image processing apparatus for generating an assist image that is an image providing guidance in moving an instrument to a target part of a subject, comprising:

a three-dimensional image analyzer determining, as target position information, a three-dimensional position of the target part based on a three-dimensional image including the target part;

a position information acquirer acquiring instrument position information indicating a three-dimensional position of the instrument;

a display state determiner selecting one display state from at least two display states based on a positional relationship between the target part and the instrument;

an assist image generator generating an assist image for the selected display state by using the target position information and the instrument position information; and

a display controller performing control for outputting the assist image generated by the assist image generator to a display device.

2. The image processing apparatus according to claim 1, wherein:

the at least two display states include;

a first display state where the assist image generated by the assist image generator is displayed at a first magnification ratio, and

a second display state where the assist image generated by the assist image generator is displayed at a second magnification ratio greater than the first magnification ratio, and

the display state determiner selects the first display state when the positional relationship does not fulfill a first predetermined condition, and selects the second display state when the positional relationship fulfills the first predetermined condition.

3. The image processing apparatus according to claim 1, wherein:

the three-dimensional image analyzer determines, as the target position information, an orientation of the target part based on the three-dimensional image, in addition to determining the three-dimensional position of the target part as the target position information, and

the position information acquirer acquires, as the instrument position information, an orientation of the instrument, in addition to the three-dimensional position of the instrument.

4. The image processing apparatus according to claim 3, wherein:

the instrument is a probe in an ultrasound diagnostic device, the probe usable for acquiring an ultrasound image of the subject,

the position information acquirer acquires, as the instrument position information, a scan position and an orientation of the probe, and

the assist image generated by the assist image generator is an image providing guidance in moving the probe to the target part.

5. The image processing apparatus according to claim 4, further comprising:

a live image acquirer acquiring, from the probe, the ultrasound image of the subject as a live image, wherein the display controller outputs the assist image generated by the assist image generator and the live image to the display device.

6. The image processing apparatus according to claim 5, wherein:

the at least two display states include;

a third display state where on the display device, the assist image generated by the assist image generator is displayed as a main image and the live image is displayed as a sub image, the sub image smaller than the main image, and

a fourth display state where on the display device, the live image is displayed as the main image and the assist image generated by the assist image generator is displayed as the sub image,

the display state determiner selects the third display state when the positional relationship does not fulfill a second predetermined condition, and selects the fourth display state when the positional relationship fulfills the second predetermined condition, and

the display controller outputs the assist image generated by the assist image generator and the live image to the display device so as to be displayed in the selected display state.

7. The image processing apparatus according to claim 6, wherein the display controller outputs the assist image generated by the assist image generator and the live image to the display device while, based on the selected display state, changing relative sizes at which the assist image generated by the assist image generator and the live image are to be displayed and thereby exchanging the main image and the sub image.

8. The image processing apparatus according to claim 6, wherein when the third display state is currently selected, the display state determiner selects the display state based on whether the positional relationship fulfills a third predetermined condition, and when the fourth display state is currently selected, the display state determiner selects the display state based on whether the positional relationship fulfills a fourth predetermined condition.

9. The image processing apparatus according to claim 5, wherein:

the target part is a blood vessel, and

the display state determiner determines the positional relationship according to whether the live image includes a cross section substantially parallel with a direction in which the blood vessel runs, and selects one of the at least two display states based on the positional relationship so determined.

10. The image processing apparatus according to claim 4, further comprising:

a three-dimensional image generator generating the three-dimensional image from data acquired in advance, wherein;

the data acquired in advance is the ultrasound image, which is obtained by the probe scanning a region including the target part, and the three-dimensional image generator extracts a contour of an organ including the target part from the ultrasound image so as to generate the three-dimensional image, and

the three-dimensional image generator associates a position and an orientation of the three-dimensional image in a three-dimensional space with the scan position and the orientation of the probe acquired by the position information acquirer.

11. The image processing apparatus according to claim 4, wherein

the assist image generator generates navigation information based on a relative relationship between a current scan position of the probe and the position of the target part, and a relative relationship between a current orientation of the probe and the orientation of the target part, and generates, as the assist image, an image in which the navigation information and a probe image indicating the current scan position and the current orientation of the probe are superimposed on the three-dimensional image.

12. The image processing apparatus according to claim 6, wherein

when the fourth display state is selected, the assist image generator generates a plurality of cross-sectional images each indicating a cross-sectional shape of the target part from one of a plurality of directions, and generates, as the assist image, an image in which a probe image indicating a current scan position and a current orientation of the probe is superimposed on each of the cross-sectional images.

13. The image processing apparatus according to claim 12, wherein:

the target part is a blood vessel,

the plurality of cross-sectional images includes two cross-sectional images, one of the two cross-sectional images indicating a cross-sectional shape of the blood vessel from a long axis direction being a direction in which the blood vessel runs, and the other one of the two cross-sectional images indicating a cross-sectional shape of the blood vessel from a short axis direction being substantially perpendicular to the long axis direction, and

the assist image generator generates, as the assist image, an image in which a straight line or a rectangle providing guidance in moving the probe to the target part is superimposed on each of the two cross-sectional images, based on a relative relationship between the current scan position of the probe and the position of the target part and a relative relationship between the current orientation of the probe and the orientation of the target part.

14. The image processing apparatus according to claim 3, wherein the display state determiner calculates, as the positional relationship, a difference between the position of the target part and the position of the instrument, and a difference between the orientation of the target part and the orientation of the instrument by using the target position information and the instrument position information, and selects one of the at least two display states according to the differences so calculated.

15. The image processing apparatus according to claim 3, wherein the display state determiner calculates a difference between the position of the target part and the position of the instrument, and a difference between the orientation of the target part and the orientation of the instrument by using the target position information and the instrument position information, and holds the differences so calculated, so as to calculate, as the positional relationship, changes occurring in the differences as time elapses and to select one of the at least two display states according to the changes in the differences so calculated.

16. The image processing apparatus according to claim 1, wherein:

the target part is a part of the subject that is a target of surgery,

the instrument is a surgical instrument used in the surgery, and

the assist image generated by the assist image generator is an image providing guidance in moving the surgical instrument to the part of the subject that is the target of surgery.

17. The image processing apparatus according to claim 16, further comprising:

a three-dimensional image generator generating the three-dimensional image from data acquired in advance.

18. The image processing apparatus according to claim 1, wherein the display state determiner calculates, as the positional relationship, a difference between the position of the target part and the position of the instrument by using the target position information and the instrument position information, and selects one of the at least two display states according to the difference so calculated.

19. The image processing apparatus according to claim 1, wherein the display state determiner calculates a difference between the position of the target part and the position of the instrument by using the target position information and the instrument position information, and holds the difference so calculated, so as to calculate, as the positional relationship, a change occurring in the difference as time elapses, and select one of the at least two display states according to the change in the difference so calculated.

20. The image processing apparatus according to claim 1, wherein:

the at least two display states include two or more display states differing from one another in terms of at least one of a modification ratio and a viewpoint of the assist image, and

the display state determiner selects one of the two or more display states based on the positional relationship.

21. An image processing method for generating an assist image that is an image providing guidance in moving an instrument to a target part of a subject, comprising:

determining, as target position information, a three-dimensional position of the target part based on a three-dimensional image including the target part;

acquiring instrument position information indicating a three-dimensional position of the instrument;

selecting one display state from at least two display states based on a positional relationship between the target part and the instrument;

generating an assist image for the selected display state by using the target position information and the instrument position information; and

performing control for outputting the assist image generated by the assist image generator to a display device.

22. A non-transitory computer-readable recording medium having recorded thereon a program for generating an assist image that is an image providing guidance in moving an instrument to a target part of a subject, the program causing a computer to execute:

determining, as target position information, of a three-dimensional position of the target part based on a three-dimensional image including the target part;

acquiring of instrument position information indicating a three-dimensional position of the instrument;

selecting of one display state from at least two display states based on a positional relationship between the target part and the instrument;

generating of an assist image for the selected display state by using the target position information and the instrument position information; and

performing of control for outputting the assist image generated by the assist image generator to a display device.