Methods and systems for performing segmentation and registration of images using neutrosophic similarity scores

• US 10,152,796 B2
• Filed: 02/24/2015
• Issued: 12/11/2018
• Est. Priority Date: 02/24/2014
• Status: Active Grant

• Current Assignee: H Lee Moffitt Cancer Center & Research Institute (State University System of Florida)
• Original Assignee: H Lee Moffitt Cancer Center & Research Institute (State University System of Florida)
• Inventors: Guo, Yanhui, Gonzalez, Segundo Jaime
• Primary Examiner(s): Park, Edward
• Application Number:
  US15/121,246
• Publication Number:
  US 20160364878A1
• Time in Patent Office: 1,386 Days
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/502 Headgear, e.g. helmet, spectacles
• CPC Class Codes: A61B 2576/00 Medical imaging apparatus involving image processing or analysis A61B1/00009, A61B6/52 and A61B8/52 take precedence; image processing per se G06T; details of algorithms for analysing medical images G06T7/0012
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10136 3D ultrasound image
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

An example method for segmenting an object contained in an image includes receiving an image including a plurality of pixels, transforming a plurality of characteristics of a pixel into respective neutrosophic set domains, calculating a neutrosophic similarity score for the pixel based on the respective neutrosophic set domains for the characteristics of the pixel, segmenting an object from background of the image using a region growing algorithm based on the neutrosophic similarity score for the pixel, and receiving a margin adjustment related to the object segmented from the background of the image.

Claims

1. A method for registering a plurality of images containing an object, comprising:

receiving, using at least one processor, a first image including a plurality of pixels;

calculating, using the at least one processor, respective first neutrosophic similarity scores for each of the pixels of the first image;

segmenting, using the at least one processor, an object from background of the first image using a region growing algorithm based on the respective first neutrosophic similarity scores for each of the pixels;

receiving, using the at least one processor, a margin adjustment related to the object segmented from the background of the first image;

receiving, using the at least one processor, a second image including a plurality of pixels;

calculating, using the at least one processor, respective second neutrosophic similarity scores for each of the pixels of the second image;

performing, using the at least one processor, a template matching algorithm based on differences between the respective first and second neutrosophic similarity scores for each of the pixels of the first and second images, respectively, to determine one or more registration parameters;

registering, using the at least one processor, the first and second images using the one or more registration parameters;

receiving, using the at least one processor, an annotation related to the object segmented from the background of the first image;

storing, using the at least one processor, the annotation related to the object segmented from the background of the first image;

overlaying the annotation relative to the object segmented from background of the second image;

transmitting, using the at least one processor, the second image with the overlaid annotation to an augmented reality (“

AR”

) head-mounted device; and

displaying, using the AR head-mounted device, the second image with the overlaid annotation.

2. The method of claim 1, wherein the one or more registration parameters are determined by minimizing the differences between the respective first and second neutrosophic similarity scores for each of the pixels of the first and second images, respectively.

3. The method of claim 1, further comprising segmenting, using the at least one processor, the object from the background in the second image using the region growing algorithm based on the respective second neutrosophic similarity scores for each of the pixels.

4. The method of claim 3, further comprising receiving, using the at least one processor, a margin adjustment related to the object segmented from the background of the second image.

5. The method of claim 3, wherein a pixel is merged into a region containing the object under the condition that the respective first or second neutrosophic similarity score for the pixel is less than a threshold value.

6. The method of claim 3, wherein a pixel is merged into a region containing the background under the condition that the respective first or second neutrosophic similarity score for the pixel is greater than a threshold value.

7. The method of claim 3, wherein calculating the respective first or second neutrosophic similarity scores for each of the pixels of the first or second image further comprises:

transforming, using the at least one processor, a plurality of characteristics of each of the pixels of the first or second image into respective neutrosophic set domains; and

calculating the respective first or second neutrosophic similarity scores for each of the pixels based on the respective neutrosophic set domains for the characteristics of each of the pixels.

8. The method of claim 7, wherein the plurality of characteristics include at least one of a respective intensity, a respective textural value and a respective homogeneity of each of the pixels.

9. The method of claim 8, wherein calculating the respective first or second neutrosophic similarity scores for each of the pixels based on the respective neutrosophic set domains for the characteristics of each of the pixels further comprises:

calculating respective first or second neutrosophic similarity scores for each of the respective neutrosophic set domains; and

calculating a mean of the respective first or second neutrosophic similarity scores for each of the respective neutrosophic set domains.

10. The method of claim 8, wherein the respective intensity of each of the pixels is transformed into an intensity neutrosophic set domain based on a respective intensity value.

11. The method of claim 8, wherein the respective homogeneity of each of the pixels is transformed into a homogeneity neutrosophic set domain based on a respective homogeneity value.

12. The method of claim 11, further comprising filtering the first or second image to obtain the respective homogeneity of the pixel.

13. The method of claim 7, wherein each of the respective neutrosophic set domains includes a true value, an indeterminate value and a false value.

14. The method of claim 1, further comprising:

receiving, using the at least one processor, information regarding a user'"'"'s movement from the AR head-mounted device;

adjusting, using the at least one processor, at least one of a position or an orientation of the second image with the overlaid annotation; and

transmitting, using the at least one processor, the adjusted second image with the overlaid annotation to the AR head-mounted device.

15. The method of claim 1, wherein the first image is a pre-operative image and the second image is a real-time, intra-operative image.

16. The method of claim 1, wherein each of the first and second images provides a 2D or 3D visualization of the object.

17. The method of claim 1, wherein the object includes a lesion region of interest.

18. The method of claim 1, wherein the at least one processor is part of a cloud computing environment.