Method and system for processing regions of interest for objects comprising biological material

• US 20040085443A1
• Filed: 11/24/2003
• Published: 05/06/2004
• Est. Priority Date: 12/13/2000
• Status: Abandoned Application

• Current Assignee: The United States of America As Represented By The Secretary of Agriculture
• Original Assignee: The United States of America As Represented By The Secretary of Agriculture
• Inventors: Karareka, John, Pohida, Thomas J, Salem, Ghadi Hamdi, Kallioniemi, Olli P
• Application Number:
  US10/450,372
• Publication Number:
  US 20040085443A1
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G01N 1/36 Embedding or analogous mounting of samples
• CPC Class Codes: G01N 2001/282 with mapping; Identification of areas; Spatial correlated pattern
• CPC Class Codes: G01N 2001/368 Mounting multiple samples in one block, e.g. TMA [Tissue Microarrays]
• CPC Class Codes: G06T 2207/30072 Microarray; Biochip, DNA array; Well plate
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/693 Acquisition
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method and apparatus are disclosed for processing regions of interest for objects comprising biological material. A region of interest can be denoted for a physical object and information indicating the region of interest can be stored in a computer-readable medium for later retrieval. Subsequently, when the object is retrieved, the information indicating the region of interest can be used to generate information specifying a physical location within the region of interest. An operation can then be performed on the physical location within the region of interest. Reference pints within the object can assist in regeneration of the region of interest, and the reference points can be arranged in such a fashion that processing can take rotation of the object into account. The invention includes various features advantageous for constructing tissue microarrays.

Claims

1. A computer-implemented method for automated processing of a physical object comprising biological material, the method comprising:

retrieving information indicating at least one region of interest of the physical object comprising biological material; and

based at least on the information indicating the region of interest, performing an operation on a physical location within the region of interest of the physical object comprising biological material.

2. The method of claim 1 wherein the information indicating the region of interest is independent of rotation of the physical object.

3. The method of claim 1 wherein the operation physically modifies the region of interest.

4. The method of claim 3 further comprising:

updating a database to reflect the region of interest has been modified.

5. The method of claim 4 further comprising:

after modifying the region of interest and updating the database, consulting the database to determine whether the region of interest meets specified criteria; and

responsive to determining the region of interest meets the specified criteria, adding the region of interest to a list of selected regions of interest on which the operation is to be performed.

6. The method of claim 1 wherein the operation comprises extracting material from the region of interest with an automated extractor.

7. The method of claim 6 further comprising:

inserting a filler material into the region of interest into an area from which the material has been extracted.

8. The method of claim 1 wherein the operation comprises removing biological tissue from the region of interest with an automated tissue punch.

9. The method of claim 1 wherein performing the operation comprises sending directives to an automated positioning device.

10. The method of claim 1 further comprising:

based at least in part on where in physical space the physical object is positioned, determining a translation for converting a location in coordinates related to the information indicating a region of interest of the physical object into coordinates indicating a location in physical space within the region of interest;

wherein performing the operation comprises applying the translation.

11. The method of claim 1 further comprising:

based at least in part on where in physical space the physical object is positioned, determining a translation for converting a location in coordinates related to the information indicating a region of interest of the physical object into coordinates related to an automated positioning device;

wherein performing the operation comprises applying the translation.

12. The method of claim 11 further comprising:

based on the coordinates related to the automated positioning device, sending directives to the automated positioning device to position the object so the operation can be performed at a physical location within the region of interest of the physical object.

13. The method of claim 11 wherein

the capturing comprises capturing an image of an item at a known location;

and

the translation is based at least in part on the location within the image of the item at the known location.

14. The method of claim 13 further comprising:

retrieving the physical object; and

before capturing the image, placing the physical object at a location adjacent to the item at the known location.

15. The method of claim 13 wherein the item at the known location is a system reference point.

16. The method of claim 13 wherein the operation comprises extracting material from the region of interest with an automated extractor, the method further comprising:

positioning the automated extractor so it extracts material from the known location.

17. The method of claim 13 wherein

the item is on a platform including at least one laser beam;

and

the known location of the item is determined by positioning the platform so that the laser beam is broken.

18. The method of claim 1 wherein

the physical object rests on a platform;

and

performing the operation comprises sending directives to an automated positioning device to move the platform, thereby positioning the physical object at a location appropriate for performing the operation on the region of interest of the physical object.

19. The method of claim 1 further comprising determining where in physical space the object is positioned with respect to a known position in physical space by capturing an image of the object;

wherein the operation is performed based at least on part on where in physical space the object is positioned with respect to the known position in physical space.

20. The method of claim 1 wherein

the physical object comprises one or more reference points;

and

the information indicating the region of interest of the physical object indicates the region of interest with respect to the reference points, the method further comprising;

capturing an image representative of the object;

determining the locations of the reference points on the image of the object; and

reconstructing the region of interest based on the locations of the reference points and the information indicating the region of interest with respect to the reference points;

wherein the operation is performed based on the reconstructed region of interest.

21. The method of claim 20 wherein

the information indicating the region of interest specifies a set of points forming a perimeter of the region of interest via distances from the reference points;

and

the information indicating the region of interest further specifies information for resolving ambiguity when one of the reference points cannot be located.

22. The method of claim 20 wherein

the information indicating the region of interest specifies a set of points forming a perimeter of the region of interest;

and

the information indicating the region of interest further specifies whether the points are below or above a line connecting sets of two of the reference points.

23. The method of claim 1 further comprising:

with the information indicating the region of interest, generating information specifying a physical location within the region of interest;

wherein the operation is performed via the information specifying the physical location within the region of interest.

24. The method of claim 1 further comprising:

determining a scaling factor for adjusting the size of the region of interest indicated by the information indicating the region of interest; and

applying the scaling factor to adjust the size of the region of interest.

25. The method of claim 24 wherein the information indicating the region of interest indicates a stored distance between two reference points of the physical object, the method further comprising:

capturing an image of the object; and

determining an observed distance between the two reference points;

wherein the scaling factor is determined based on the stored distance and the observed distance.

26. The method of claim 1 further comprising:

retrieving the physical object via an automated object retriever.

27. The method of claim 1 wherein

the object is a donor tissue block;

and

the operation comprises extracting tissue from the region of interest of the donor tissue block.

69. (New) A computer-readable medium comprising computer-executable instructions for performing the method of claim 4.

70. (New) A computer-readable medium comprising computer-executable instructions for performing the method of claim 19.

71. (New) A computer-readable medium comprising computer-executable instructions for performing the method of claim 20.

28. A computer-readable medium comprising computer-readable instructions for performing the following to process a physical object comprising biological material:

retrieving information indicating at least one region of interest of the physical object comprising biological material; and

based at least on the information indicating the region of interest, performing an operation on a physical location within the region of interest of the physical object comprising biological material.

72. (New) The computer-readable medium of claim 28 wherein performing an operation comprises sending directives to an automated positioning device.

29. A computer-implemented method for processing an observable feature comprising biological material in a physical object, the method comprising:

capturing a first image depicting the observable feature comprising biological material;

via the first image, denoting at least one region of interest comprising the feature comprising biological material;

storing information indicating the region of interest;

retrieving the information indicating the region of interest;

capturing a second image, wherein the second image depicts an item of known location and the object;

based on the second image and the retrieved information indicating the region of interest, generating information to position the feature comprising biological material at a location appropriate for extracting material from the feature comprising biological material;

sending the information to an automated positioning device to position the feature comprising biological material at a location appropriate for extracting material from the feature; and

extracting material from the feature comprising biological material.

30. The method of claim 29 wherein the first image is of a first magnification and the second image is of a second, different magnification.

31. The method of claim 29 wherein the physical object is a tissue block, and the feature present in the physical object is a region of tissue of a particular tissue type.

32. A computer-implemented method for processing a physical object comprising biological material, the method comprising:

during a first session, capturing a first image representative of the physical object comprising biological material;

during the first session, designating one or more regions of interest for the object comprising biological material via the first captured image;

during the first session, storing information indicating the one or more regions of interest for the physical object comprising biological material;

during a second, subsequent session, retrieving the physical object comprising biological material;

during the second, subsequent session, retrieving the information indicating the one or more regions of interest for the physical object comprising biological material;

during the second, subsequent session, capturing a second image of the physical object comprising biological material;

during the second, subsequent session, based on the second captured image and the retrieved information indicating the one or more regions of interest for the physical object comprising biological material, performing an operation on one or more physical locations within the one or more regions of interest for the physical object comprising biological material.

33. The method of claim 32 further comprising:

reconstructing location and extent of the one or more regions of interest from the information indicating the one or more regions of interest for the physical object;

wherein the operation is performed via the location and extent of the one or more regions of interest.

34. The method of claim 32 further comprising:

reconstructing perimeters of the one or more regions of interest from the information indicating the one or more regions of interest for the physical object;

wherein the operation is performed via the perimeters of the one or more regions of interest.

35. The method of claim 32 further comprising:

correcting error in the information indicating the one or more regions of interest for the physical object based on overlap between regions shown on the first image and regions shown on the second image.

36. A computer-implemented method for processing regions of interest in a set of physical objects comprising biological material, the method comprising:

denoting a plurality of regions of interest for the physical objects comprising biological material;

selecting a list of a subset of the plurality of regions of interest; and

for the regions of interest appearing on the list, performing the following;

automatically retrieving a physical object comprising biological material having the region of interest; and

automatically extracting material from the region of interest of the physical object comprising biological material.

37. The computer-implemented method of claim 36 wherein the selecting comprises performing a database query on a database storing information about the physical objects and the regions of interest.

38. The computer-implemented method of claim 36 wherein the selecting is performed from a location remote from where the automatically retrieving and automatically extracting are performed.

39. A computer-implemented method for processing a physical object comprising biological material, wherein the physical object comprises a plurality of reference points indicated thereon, the method comprising:

retrieving information indicating a region of interest on the physical object comprising biological material with respect to the reference points;

capturing an image representing the physical object comprising biological material, the object'"'"'s reference points, and one or more system reference points;

finding locations of the object'"'"'s reference points and the system reference points on the image;

calculating a translation mapping location on the image to absolute locations sufficient to position a robotic arm at a physical location of the physical object corresponding to the locations on the image;

choosing a location within the region of interest;

with the translation, mapping the chosen location to physical location information sufficient to position an automated device at a physical location corresponding to the chosen location;

sending the physical location information to position the automated device at the physical location corresponding to the chosen location; and

with the automated device, performing an operation on the physical location within the region of interest of the physical object comprising biological material.

40. A computer-implemented method for denoting one or more regions of interest for a physical object comprising biological material, wherein the physical object comprises a plurality of reference points, the method comprising:

capturing an image representative of the physical object comprising biological material, wherein the image depicts locations of at least one of the reference points;

via the captured image, denoting one or more regions of interest for the physical object comprising biological material; and

in a computer-readable medium, storing information indicating the one or more regions of interest with respect to the reference points.

41. The method of claim 40 wherein

the image representative of the physical object depicts a slice taken from the physical object;

and

the locations of the reference points on the slice are associated with the locations of the reference points on the physical object.

42. The method of claim 40 wherein the information indicating the one or more regions of interest indicates the regions of interest by indicating the location and extent of the regions of interest with respect to the reference points.

43. The method of claim 40 further comprising:

determining the area of at least one of the regions of interest; and

based on the area, storing information indicating an available area for the region of interest;

wherein the information indicating the available area is stored with the information indicating the regions of interest.

44. The method of claim 43 further comprising:

removing material from the region of interest; and

adjusting the stored available area for the region of interest to reflect material has been removed from the region of interest.

45. The method of claim 40 wherein the denoting comprises automated tracing of a perimeter physically appearing on the physical object.

46. The method of claim 40 wherein the denoting comprises tracing by an operator of a perimeter physically appearing on the physical object and depicted on the image.

47. The method of claim 40 further comprising:

collecting information indicating a scale of the reference points; and

based on the information indicating scale, calculating a size of the region of interest.

48. The method of claim 40 wherein

identifiers are assigned to the reference points;

and

the reference points are placed at locations on the physical object whereby the assigned identifier can be determined based on the location of the reference points with respect to features of the object.

49. The method of claim 40 wherein the information indicating the region of interest is in a format independent of whether the object is rotated.

50. The method of claim 40 wherein placement of the reference points permits determining the orientation of the physical object based on the location of the reference points, even if the physical object has been rotated or inverted.

51. The method of claim 40 further comprising:

after storing the information, presenting a user interface by which an operator can adjust location and extent of the region of interest.

52. The method of claim 40 wherein the information indicating the region of interest comprises information sufficient to choose between ambiguous results for the region of interest if one of the reference points cannot be located.

53. The method of claim 52 wherein

the information indicating the region of interest specifies a set of points forming a perimeter of the region of interest;

and

the information indicating the region of interest further specifies whether the points are below or above a line connecting sets of two of the reference points.

54. The method of claim 40 wherein the information indicating the region of interest indicates a perimeter of the region of interest by designating a set of points, the points designated by specifying distances between the points and the reference points.

55. The method of claim 40 wherein the physical object is a tissue block.

56. A computer-implemented method of translocating tissue from a donor tissue block to a recipient block, the method comprising:

capturing a first image of a slice taken from the donor tissue block;

based on the first image, storing information indicative of a region of interest for the donor tissue block;

capturing an image of the donor tissue block;

based on the second image and the information indicative of a region of interest, regenerating the region of interest for the donor tissue block; and

via automated means, directing a tissue punch to a location appropriate for punching tissue from the region of interest of the donor tissue block;

with the tissue punch, punching tissue from the region of interest of the donor tissue block;

via automated means, directing the tissue punch to a location appropriate for depositing the tissue into the recipient block; and

with the tissue punch, depositing the tissue from the region of interest of the donor tissue block into the recipient block.

57. A computer-readable medium comprising a data structure indicating a region of interest on a physical object comprising biological material, the data structure comprising:

information indicating locations of a plurality of reference points on the physical object comprising biological material; and

information indicating a location and extent of a region of interest with respect to the reference points;

whereby the data structure, when processed by an automated system, causes regeneration of the region of interest.

58. The computer-readable medium of claim 57 wherein the location of the region of interest with respect to the reference points is indicated by specifying points on a border of the region of interest according to their distances from the reference points.

59. The computer-readable medium of claim 58 further comprising:

information to differentiate between plural possible locations of a point on the border when the distances from the reference points are ambiguous.

60. A computer-implemented method for processing a plurality of tissue types, the method comprising:

for a plurality of donor tissue blocks, denoting regions of interest on captured images of the donor tissue blocks, wherein the regions of interest comprise tissue of interest of a particular tissue type;

submitting a list of a subset of the regions of interest to an automated arrayer, wherein the list represents a plurality of tissue types; and

with the automated arrayer, based on the list, extracting tissue from the regions of interest and collecting them into a recipient tissue block to produce a recipient tissue block having each tissue type represented on the list.

61. The method of claim 60 wherein software chooses a location within the region of interest from which tissue is to be extracted.

62. The method of claim 60 further comprising:

generating the list by querying a database for regions of interest having characteristics satisfying specified criteria.

63. The method of claim 62 wherein the criteria include a requirement that at least a certain amount of tissue is available.

64. The method of claim 62 wherein the criteria include a requirement that the region of interest be at least a certain distance from a specified feature.

65. An automated tissue microarray construction system comprising:

an image capturing device operable to capture an image;

an automated tissue block retriever operable to retrieve one of a plurality of tissue blocks; and

a computer system operable to receive a captured image from the image capturing device;

wherein the computer system is further operable to accept a list of regions of interest from which material is to be extracted, retrieve the tissue blocks corresponding to the regions of interest, capture images of the retrieved blocks, and based on the images, extract tissue from the regions of interest appearing on the list.

66. The system of claim 65 wherein the computer system comprises a database storing information indicating information for the blocks, the regions of interest, and relationships between the blocks and the regions of interest.

67. An automated tissue microarray construction system comprising:

automated means for accepting designation of one or more regions of interest for a plurality of donor objects comprising tissue to be arrayed on one or more recipient objects; and

automated means for retrieving tissue from within the regions of interest of a plurality of donor objects and placing the tissue within one of the recipient objects.

68. The automated tissue microarray construction system of claim 67 further comprising:

storage means for tracking an amount of tissue available within the regions of interest for the plurality of donor objects.