Virtual endoscopic image generated using an opacity curve

• US 9,865,079 B2
• Filed: 01/31/2011
• Issued: 01/09/2018
• Est. Priority Date: 03/31/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Miyamoto, Masaki
• Primary Examiner(s): Drennan, Barry
• Assistant Examiner(s): PRINGLE-PARKER, JASON A
• Application Number:
  US13/017,302
• Publication Number:
  US 20110242097A1
• Time in Patent Office: 2,535 Days
• Field of Search: None
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 15/20 Perspective computation

Abstract

In generating a virtual endoscopic image, an interior of a lumen is made viewable even when the viewpoint position is changed. A virtual endoscopic image generation element generates a virtual endoscopic image by volume rendering based on three-dimensional data. An opacity curve setting element sets an opacity curve which defines the relationship between pixel values of the three-dimensional data and opacity values. A viewpoint position setting element sets a viewpoint position of a virtual endoscopic image. A movement amount determination element determines a movement amount of the opacity curve with respect to an opacity curve at a reference viewpoint position. When a virtual endoscopic image is generated by the virtual endoscopic image generation element, the opacity curve setting element sets an opacity curve obtained by moving the opacity curve by the determined movement amount in the virtual endoscopic image generation element.

Claims

1. A projection image generation apparatus, comprising:

a virtual endoscopic image generator that generates, based on three dimensional data representing an interior of a subject having a lumen captured by an imaging device, a virtual endoscopic image, which is a pseudo three dimensional image, for visualizing an interior of the lumen by volume rendering;

an opacity curve setter that sets, in the virtual endoscopic image generator, an opacity curve which defines a relationship between pixel values of the three dimensional data and opacity values and is used in the volume rendering;

a viewpoint position setter that sets a viewpoint position corresponding to a camera position of the virtual endoscopic image within the interior of the lumen in the virtual endoscopic image generator, and is capable of setting different viewpoints in the extension direction of the lumen; and

a movement amount determination device that determines a movement amount of the opacity curve for the viewpoint position set by the viewpoint position setter with respect to the opacity curve at a reference viewpoint position in the case that the viewpoint position set by the viewpoint position setter is changed in the extension direction of the lumen from the reference viewpoint position, after the change in the viewpoint position,

wherein a viewpoint position change is made by the viewpoint position setter, the opacity curve setter automatically moves the opacity curve at the reference viewpoint position by the movement amount determined by the movement amount determination device for the viewpoint position after the change, and sets the moved opacity curve in the virtual endoscopic image generator.

2. The projection image generation apparatus of claim 1, wherein the opacity curve setter translates a relationship between pixel values and opacity values defined by the opacity curve at the reference viewpoint position in a pixel value direction by the determined movement amount.

3. The projection image generation apparatus of claim 1, wherein, when pixel values are represented by a variable “

v”

, the opacity curve at the reference viewpoint position is represented by O_D(v), and the movement amount determined by the movement amount determination device is represented by “

m”

, the opacity curve setter sets an opacity curve represented by O(v)=O_D(v−

m) in the virtual endoscopic image generation device.

4. The projection image generation apparatus of claim 1, wherein the reference viewpoint position is a predetermined initial viewpoint position.

5. The projection image generation apparatus of claim 1, wherein, when a viewpoint position change is made by the viewpoint position setter, the reference viewpoint position is the viewpoint position before the change.

6. The projection image generation apparatus of claim 1, wherein the movement amount determination device determines the movement amount based on a data distribution of the three-dimensional data in an adjacent area of the reference viewpoint position and a data distribution of the three-dimensional data in an adjacent area of the viewpoint position set by the viewpoint position setter.

7. The projection image generation apparatus of claim 1, wherein the movement amount determination device determines the movement amount by performing matching between a histogram representing a data distribution of the three-dimensional data in an adjacent area of the reference viewpoint position and a histogram representing a data distribution of the three-dimensional data in an adjacent area of the viewpoint position set by the viewpoint position setter.

8. The projection image generation apparatus of claim 7, wherein, when the histogram of pixel values of the three-dimensional data in an adjacent area of the reference viewpoint position is H_D(v) with the pixel values being represented by a variable “

v” and

the histogram of pixel values of the three-dimensional data in an adjacent area of the viewpoint position set by the viewpoint position setter is H_n(v) with the pixel values being represented by the variable “

v”

, the movement amount determination device calculates a similarity degree between a histogram represented by H_D(v−

x) and the histogram H_n(v) by changing the value of “

x” and

determines a value of “

x”

when the similarity degree becomes maximum as the movement amount.

9. The projection image generation apparatus of claim 7, wherein, when the histogram of pixel values of the three-dimensional data in an adjacent area of the reference viewpoint position is H_D(v) with the pixel values being represented by a variable “

v”

, the histogram of pixel values of the three-dimensional data in an adjacent area of the viewpoint position set by the viewpoint position setter is H_n(v) with the pixel values being represented by the variable “

v”

, and P(H(v)) is a function for obtaining a peak position in histogram H(v), the movement amount determination device determines a value of “

x”

obtained by x=P(H_D(v))—

P(H_n(v)) as the movement amount.

10. The projection image generation apparatus of claim 4, wherein:

the apparatus further comprises a control point setter that sets one or more control points in the lumen, setting an opacity curve for each control point by translating the opacity curve at the initial viewpoint position in a pixel value direction, and storing a location of each control point in the lumen and a movement amount from the opacity curve at the initial viewpoint position associated with each other as control point information; and

the movement amount determination device obtains an estimated value of movement amount at the viewpoint position set by the viewpoint position setter from the opacity curve at the initial viewpoint position based on the location of the viewpoint position in the lumen set by the viewpoint position setter, and the location of a control point in the lumen and the movement amount from the opacity curve at the initial viewpoint position included in the control point information.

11. The projection image generation apparatus of claim 10, wherein the apparatus further comprises a path setter that sets a path in the lumen, and the locations of the viewpoint position and each control point in the lumen are represented by distances from a starting point of the path.

12. The projection image generation apparatus of claim 10, wherein:

the control point setter sets two or more control points; and

the movement amount determination device obtains, when the viewpoint position set by the viewpoint position setter is located between two control points in the lumen, an estimated value of movement amount at the viewpoint position set by the viewpoint position setter by interpolation based on the movement amount of each of at least the two control points.

13. The projection image generation apparatus of claim 1, wherein:

the apparatus further comprises a color map setter that sets a color map which defines the relationship between pixel values of the three-dimensional data and display colors, and is used in the volume rendering; and

when a viewpoint position change is made by the viewpoint position setter the color map setter automatically moves a color map at the reference viewpoint position by the movement amount determined by the movement amount determination device for the viewpoint position after the change, and sets the moved color map in the virtual endoscopic image generator.

14. The projection image generation apparatus of claim 13, wherein the color map setter translates the relationship between pixel values and display colors defined by the color map at the reference viewpoint position in a pixel value direction by the determined movement amount.

15. The projection image generation apparatus of claim 13, wherein, when the pixel values are represented by a variable “

v”

, the color map at the reference viewpoint position is represented by ClrM_apD (v), and the movement amount determined by the movement amount determination device is represented by “

m”

, the color map setter sets a color map represented by ClrM_ap(v)=ClrM_apD (v−

m) to the virtual endoscopic image generator.

16. A projection image generation method, comprising using a computer to perform the steps of:

generating based on three-dimensional data representing an interior of a subject having a lumen captured by an imaging device, a virtual endoscopic image, which is a pseudo three-dimensional image, for visualizing an interior of the lumen by volume rendering; and

displaying the virtual endoscopic image on a display,

wherein when a viewpoint position of the virtual endoscopic image, which corresponds to a camera position of the virtual endoscopic image within the interior of the lumen and is capable of setting different viewpoints in the extension direction of the lumen, is changed, an opacity curve defines a relationship between pixel values of the three-dimensional data and opacity values, and is used in the volume rendering, is automatically moved from the opacity curve at a reference viewpoint position in the case that the viewpoint position set by the viewpoint position setter is changed in the extension direction of the lumen from the reference viewpoint position, after the change in the viewpoint position in accordance with the change of the viewpoint position and the moved opacity curve is used to generate the virtual endoscopic image at the viewpoint position after the change.

17. The projection image generation method of claim 16, wherein the movement of the opacity curve is implemented by determining a movement amount for a viewpoint position at which a virtual endoscopic image is to be generated from the opacity curve at the reference viewpoint position, and moving the opacity curve by the determined movement amount.

18. The projection image generation method of claim 17, wherein the movement amount of the opacity curve is determined based on a data distribution of the three-dimensional data in an adjacent area of the reference viewpoint position and a data distribution of the three-dimensional data in an adjacent area of the viewpoint position at which a virtual endoscopic image is to be generated.

19. The projection image generation method of claim 17, wherein:

before generating the virtual endoscopic image, one or more control points are set in the lumen, a movement amount of the opacity curve for each control point is set, and a location of each control point in the lumen and the movement amount of the opacity curve are associated with each other and stored as control point information; and

the movement amount of the opacity curve is determined by obtaining an estimated value of movement amount for an opacity curve at the viewpoint position at which a virtual endoscopic image is to be generated based on the location of the viewpoint position in the lumen at which a virtual endoscopic image is to be generated, and the location of a control point and the movement amount of the opacity curve included in the control point information.

20. A non-transitory computer readable recording medium on which is recorded a program for causing a computer to perform the step of generating, based on three-dimensional data representing an interior of a subject having a lumen captured by an imaging device, a virtual endoscopic image within an interior of the lumen and is capable of setting different viewpoints in the extension direction of the lumen, which is a pseudo three-dimensional image, for visualizing the interior of the lumen by volume rendering,

wherein the program causes the computer to perform the step of moving an opacity curve which defines the relationship between pixel values of the three-dimensional data and opacity values, and is used in the volume rendering from an opacity curve at a reference viewpoint position in the case that the viewpoint position is changed in the extension direction of the lumen from the reference viewpoint position, after the change in the viewpoint position, and corresponding to a camera position of the virtual endoscopic image, and setting the moved opacity curve as the opacity curve used for generating the virtual endoscopic image.