Morphology identification in tissue samples based on comparison to named feature vectors

• US 9,870,615 B2
• Filed: 07/19/2017
• Issued: 01/16/2018
• Est. Priority Date: 07/19/2017
• Status: Active Grant

• Current Assignee: Schwalb Consulting, LLC
• Original Assignee: Schwalb Consulting, LLC
• Inventors: Schwalb, Edward M, Schwalb, Tammy April
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US15/653,579
• Publication Number:
  US 20170323445A1
• Time in Patent Office: 181 Days
• Field of Search: 382100, 382103, 382106-107, 382128-134, 382154, 382162, 382168, 382173, 382181, 382194, 382219, 382224, 382232, 382254-260, 382274-276, 382286-291, 382305, 382312, 703 11, 348239, 378 4, 378 43
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 2207/10056 Microscopic image
• CPC Class Codes: G06T 2207/10141 Special mode during image acquisition
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30024 Cell structures in vitro; Tissue sections in vitro
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/0014 using an image reference approach
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 20/693 Acquisition
• CPC Class Codes: G06V 20/695 Preprocessing, e.g. image segmentation
• CPC Class Codes: G06V 20/698 Matching; Classification

Abstract

Locating morphology in a tissue sample is achieved with devices and methods involving storage of a plurality of feature vectors, each associated with a specific named superpixel of a larger image of a tissue sample from a mammalian body. A microscope outputs, in some embodiments, a live image of an additional tissue sample or a digitized version of the output is used. At least one superpixel of the image is converted into a feature vector and a nearest match between the first feature vector and the plurality of stored feature vectors is made. A first name suggestion is then made based on the nearest match comparison to a store feature vector. Further, regions of interest within the image can be brought to a viewer'"'"'s attention based on their past history of selection, or that of others.

Claims

1. A method of locating morphology in a tissue sample, comprises the steps of:

obtaining a first, second, and third tissue sample, each removed from an organism;

taking a microscope to each said first, second, and third tissue sample and digitizing magnified image of said first tissue sample, said second, and said third tissue sample;

receiving a selection of at least one superpixel of said image of said first tissue sample;

receiving a named morphology for said at least one superpixel and converting said at least one superpixel into a first feature vector;

extracting superpixels of said image of said second tissue sample and converting each superpixel of said second image of said second tissue into second feature vectors;

finding, using a processor carrying out instructions, within said second feature vectors of said second tissue, a nearest match feature vector to said first feature vector; and

indicating that a superpixel of said second image associated with said nearest match feature vector is possibly a representation of said named morphology;

after receiving confirmation of a name of said named morphology from a viewer for said named morphology a threshold number of minimum times, a name of said named morphology becomes unchangeable for future scanned superpixels determined to be associated with said named morphology; and

wherein said third tissue sample has a third superpixel with a feature vector indicated as being a nearest match to a feature vector found in one of said first or said second tissue samples;

further wherein said third superpixel and corresponding superpixels associated with said match feature vector of said first or said second tissue sample are unnamed;

exhibiting at least one of said third superpixel and said corresponding superpixels;

requesting a name for said at least one of said third superpixel and said corresponding superpixels; and

providing said name at a future time when a feature vector is found corresponding to said third superpixel.

2. The method of claim 1, further comprising steps of:

exhibiting said superpixel associated with said nearest match feature vector to a viewer; and

receiving confirmation from said viewer of said superpixel associated with said nearest match feature vector is said named morphology.

3. The method of claim 1, wherein before said threshold number of minimum times, a name of said pathology is changeable for at least some feature vectors associated with a superpixel of a morphology which was previously indicated as said named morphology.

4. The method of claim 3, wherein a superpixel comprises a minimum of block size of 30 by 30 pixels and each possible block of said second image is scanned to find a superpixel which has an associated feature vector closest to said first feature vector.

5. The method of claim 1, wherein said method is carried out multiple times with a same viewer, each time said first and second tissue sample being replaced with additional said tissue samples, further comprising the steps of:

recognizing patterns of interest by said same user based on two or more of;

areas of said images where said user zooms;

areas of said image where said user pans;

areas of said images where user names regions.

6. The method of claim 5, wherein suggestions of superpixels to view are sent to said viewer based on said patterns of interest.

7. The method of claim 6, wherein said suggestions are further made for a second viewer based on said patterns of interest overlapping for said viewer and said second viewer.

8. The method of claim 6, wherein said suggestions are further made for additional viewers based on receiving an indication that said viewer and at least one additional viewer of said additional viewers is in a same medical specialty as said viewer.

9. The method of claim 1, further comprising a step of:

determining a section of an image of said second tissue sample with a high density of named morphologies;

indicating to said viewer said high density section.

10. The method of claim 9, wherein said indicating to said viewer said high density section is carried out by way of one of:

zooming in a display of said second tissue sample to said high density section;

outlining said high density section; and

color coding said high density section.

11. The method of claim 1, wherein in second tissue sample, a plurality of additional named morphologies, including said named morphology, are determined to be related to a clinical context and indicating to said viewer that a particular type of medical specialty should be employed for a patient associated with said second tissue sample.

12. The method of claim 1, wherein said second tissue sample is being viewed by said viewer in real-time using a microscope apparatus.

13. A system for locating morphology in a tissue sample, comprising:

storage of a plurality of feature vectors, each associated with a specific named superpixel of a larger image of a tissue sample from an organism;

a microscope outputting a live image of an additional tissue sample;

a processor carrying out instructions to;

a) convert at least one superpixel of said live image into a first feature vector;

b) find a nearest match between said first feature vector and said plurality of stored feature vectors; and

c) output a first name suggestion of said at least one superpixel of said live image based on said nearest match to one of said stored plurality of feature vectors associated with said specific named superpixel;

wherein the above is carried out for at least three tissue samples each with different superpixels determined to be a nearest match to said first feature vector, said first feature vector having a name which is changeable;

requesting a name for said first feature vector while exhibiting an unnamed superpixel;

providing said name at a future time when a feature vector is found corresponding to the previously said unnamed superpixel;

wherein upon receiving a confirmation of said name suggestion for said first name a threshold number of minimum times, said first name becomes permanently associated with a particular morphology and is becomes unchangeable for future scanned superpixels determined to be associated with said named morphology.

14. The system of claim 13, further comprising an input mechanism through which a recipient of said name suggestion sends said confirmation of said name suggestion.

15. The system of claim 13, wherein said at least one superpixel is a plurality of superpixels making up said live image and suggestions as to which superpixels of said plurality of superpixels to view are indicated to said viewer based on a pattern of interest identified for said viewer.

16. The system of claim 15, wherein said pattern of interest identified for said viewer is based on overlapping selections of superpixels between said viewer and a second viewer.

17. The system of claim 13, wherein said suggestions are further made for additional viewers based on receiving an indication that said viewer and at least one additional viewer of said additional viewers is in a same medical specialty as said viewer.

18. The system of claim 13, further comprising a component measuring a density of named morphologies in an image and a component exhibiting to a user relative densities of at least one selection of part of said image.

19. The system of claim 18, wherein said device determines morphologies of greatest frequency covering a plurality of superpixels and prompts said user to name said morphologies of greatest frequency.

20. The system of claim 18, wherein said device determines regions comprising a plurality of superpixels where morphologies are unnamed and prompts said user to name said regions.