Assaying and imaging system identifying traits of biological specimens

• US 20040076318A1
• Filed: 07/14/2003
• Published: 04/22/2004
• Est. Priority Date: 07/15/2002
• Status: Abandoned Application

• Current Assignee: Baylor College of Medicine (Baylor College), Envivo Pharmaceuticals, Inc., European Molecular Biology Laboratory
• Original Assignee: Baylor College of Medicine (Baylor College), Envivo Pharmaceuticals, Inc., European Molecular Biology Laboratory
• Inventors: Serrano, Luis, Cummings, Christopher J., Botas, Juan, Boulin, Christian, Faeldt, Edward, Zoghbi, Huda, Gonzalez, Cayetano
• Application Number:
  US10/618,869
• Publication Number:
  US 20040076318A1
• Time in Patent Office: Days
• US Class Current: 382/128
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

A method of system is provided for assaying specimens. In connection with such system or method, plural multi-pixel target images of a field of view are obtained at different corresponding points in time over a given sample period. A background image is obtained using a plural set of the plural target images. For a range of points in time, the background image is removed from the target images to produce corresponding background-removed target images. Analysis is performed using at least a portion of the corresponding background-removed target images to identify visible features of the specimens. A holding structure is provided to hold a set of discrete specimen containers. A positioning mechanism is provided to position a plural subset of the containers to place the moving specimens within the plural subset of the containers within a field of view of the camera.

Claims

1. A method for assaying plural biological specimens, each of the biological specimens moving within a field of view, the method comprising:

obtaining plural multi-pixel target images of the field of view at different corresponding points in time over a given sample period;

obtaining a background image using a plural set of the plural target images;

for a range of points in time, removing the background image from the target images to produce corresponding background-removed target images; and

performing analysis using at least a portion of the corresponding background-removed target images to identify visible features of the biological specimens.

2. The method according to claim 1, wherein the plural biological specimens comprise sets of biological specimens provided in discrete containers, some of the containers comprising a reference population of biological specimens and other of the containers comprising a test population of biological specimens.

3. The method according to claim 2, wherein the discrete containers comprise transparent vials.

4. The method according to claim 2, wherein the discrete containers comprise plates.

5. The method according to claim 1, wherein each of the sets of biological specimens comprises plural specimens within a discrete container.

6. The method according to claim 5, wherein the specimens comprise animals within transparent tubes.

7. The method according to claim 5, wherein the specimens comprise flies within transparent tubes.

8. The method according to claim 5, wherein the specimens comprise drosophila within transparent tubes.

9. The method according to claim 1, wherein the field of view encompasses an entire area within each of the containers that is visible to a camera.

10. The method according to claim 9, wherein the field of view captures at least a region of interest.

11. The method according to claim 1, wherein the obtaining of a background image comprises normalizing non-moving elements in the plural multi-pixel target images, the plural multi-pixel target images comprising frames of a movie.

12. The method according to claim 1, wherein the obtaining of a background image comprises removing objects from the target images by normalizing non-moving elements in the target images.

13. The method according to claim 12, wherein the normalizing comprises averaging images among a plural set of the target images.

14. The method according to claim 1, wherein the obtaining of a background image comprises superimposing two or more of the target images, and then determining a characteristic pixel value for the pixels in the superimposed target images.

15. The method according to claim 14, wherein the characteristic pixel values comprise averaged pixel values from corresponding pixels from among the plural set of target images.

16. The method according to claim 14, wherein the characteristic pixel values comprise median pixel values from corresponding pixels from among the plural set of target images.

17. The method according to claim 1, wherein the plural set comprises all of the images taken during the given sample period.

18. The method according to claim 1, wherein the removing of a background image from the target images comprises calculating a difference between the target images and the background image.

19. The method according to claim 1, further comprising further processing the background-removed target images to produce a filtered binary image.

20. The method according to claim 17, wherein the further processing comprises applying a gray-scale threshold to the background-removed target images.

21. The method according to claim 1, further comprising further processing the background-removed target images by identifying image blocks and by removing image blocks that are larger than a maximum threshold size and smaller than a minimum threshold size.

22. The method according to claim 21, wherein the maximum threshold size comprises a maximum threshold area, and wherein the minimum threshold size comprises a minimum threshold area.

23. The method according to claim 1, further comprising further processing the background-removed target images by identifying an eccentricity value for an image block and then removing image blocks that are larger than double the eccentricity value or smaller than half the eccentricity value.

24. The method according to claim 1, wherein the performing analysis comprises determining a trajectory of the biological specimens within each of the plural sets of biological specimens, the trajectory being based upon information including the orientation of a given image block representing a given biological specimen, the center of the given image block, the area of the given image block, and a velocity vector representing the velocity of the given image block.

25. The method according to claim 1, wherein the performing analysis comprises determining an orientation of the biological specimens.

26. The method according to claim 1, wherein the performing analysis comprises determining a predicted position of a given image block representing a given specimen based on previous position information regarding the given image block plus a prediction factor multiplied by a previous velocity vector.

27. The method according to claim 26, wherein the prediction factor is between zero and one.

28. The method according to claim 1, wherein the performing analysis comprises distinguishing a given specimen from other biological specimens so behavioral statistics can be correctly attributed to the given biological specimen.

29. The method according to claim 1, wherein the performing analysis comprises calculating travel distances of the biological specimens.

30. The method according to claim 29, wherein the travel distance is calculated after the biological specimens are caused to move in response to stimulation of the biological specimens.

31. The method according to claim 30, wherein the biological specimens are stimulated by subjecting them to an attraction.

32. The method according to claim 31, wherein the containers containing the specimens are moved to cause the biological specimens to move to a repeatable reference position, and wherein the biological specimens are attracted toward a given different position with light.

33. The method according to claim 1, wherein the performing analysis comprises calculating a path length of the path traveled by the specimens.

34. The method according to claim 1, wherein the performing analysis comprises calculating a speed of the biological specimens.

35. The method according to claim 1, wherein the performing analysis comprises calculating turning of the biological specimens.

36. The method according to claim 35, wherein the calculating turning of a biological specimen comprises calculating an angle between a velocity vector of a given trajectory of a biological specimen and the subsequent velocity vector of the same trajectory of the same biological specimen.

37. The method according to claim 1, wherein the performing analysis comprises calculating stumbling of a given biological specimen.

38. The method according to claim 37, wherein calculating stumbling comprises determining an angle between an orientation of an image block representing the biological specimen and a velocity vector of the image block.

39. The method according to claim 1, wherein the analysis is performed on every biological specimen of the biological specimens assayed.

40. A system for assaying plural biological specimens, each of the biological specimens moving within a field of view, the system comprising:

a holding structure to hold a set of discrete specimen containers; and

a positioning mechanism to position a plural subset of the containers to place the moving biological specimens within the plural subset of the containers within a field of view of a camera.

41. The system according to claim 40, wherein the plural biological specimens comprise sets of biological specimens provided in respective discrete containers, some of the containers comprising a reference population of biological specimens and other of the containers comprising a test population of biological specimens.

42. The system according to claim 41, wherein the discrete containers comprise transparent vials.

43. The system according to claim 41, wherein the discrete containers comprise plates.

44. The system according to claim 41, wherein each of the sets of specimens comprises plural biological specimens within a discrete container.

45. The system according to claim 44, wherein the biological specimens comprise animals within a transparent tube.

46. The system according to claim 44, wherein the biological specimens comprise flies within a transparent tube.

47. The system according to claim 44, wherein the biological specimens comprise drosophila within a transparent tube.

48. The system according to claim 40, wherein the field of view encompasses the entire area within the containers of the plural subset as visible to a camera.

49. The system according to claim 48, wherein the field of view encompasses a region of interest.

50. The system according to claim 40, wherein the holding structure comprises at least one tray of discrete specimen containers.

51. The system according to claim 40, wherein the field of view of one camera covers specimens of the plural subset.

52. The system according to claim 40, wherein one camera field of view corresponds to one container within the plural subset.

53. The system according to claim 40, wherein the containers of the plural subset are moved to an imaging position of an imaging station.

54. The system according to claim 40, wherein the positioning mechanism comprises a conveyor to move containers of the plural subset to an imaging position of an imaging station.

55. The system according to claim 40, wherein the positioning mechanism comprises a staging mechanism to move containers through positioned stages, movement from one stage to another resulting in the biological specimens being forced to a reference position, each stage corresponding to the containers being at an imaging position of an imaging station.

56. The system according to claim 55, wherein the reference position is the bottom of the container.

57. The system according to claim 40, further comprising an identification mechanism to automatically identify each container.

58. The system according to claim 57, wherein the identification mechanism comprises an identifier provided on each container, and comprises an identifier reader within a positioning path between a resting position of the container and the imaging position of the container.

59. The system according to claim 58, wherein the identifier comprises a bar code provided on each of the containers, and wherein the identifier reader comprises a bar code scanner.

60. A method of processing frames of a digitized movie comprising:

superimposing frames of the movie to obtain a background approximation; and

determining characteristic pixel values for pixels of the background approximation based on pixels of the superimposed frames.

61. The method of 60, wherein superimposing frames comprises superimposing all of the frames of the movie.

62. The method of 60, wherein superimposing frames comprises superimposing a set of frames of the movie.

63. The method of 60, wherein the characteristic values are average pixel values based on pixel values of the superimposed frames.

64. The method of 60, wherein the characteristic values are median pixel values based on pixel values of the superimposed frames.

65. The method of 60, further comprising:

subtracting the background approximation from a frame.

66. The method of 65, further comprising:

applying a gray scale threshold to create a binary image of the frame.

67. The method of 60, further comprising:

subtracting the background approximation from a first frame of the movie;

identifying a first image block in the first frame; and

assigning a first trajectory to the first image block if the first image block is within a search distance of the first trajectory.

68. The method of 67, further comprising:

identifying a second image block in a second frame of the movie;

assigning the first trajectory to the second image block if the second image block is within the search distance of the first trajectory; and

determining a velocity vector for the first trajectory based on the position of the first image block in the first frame and the position of the second image block in the second frame.

69. The method of 68, further comprising:

determining a predicted position for the first trajectory based on the location of the second image block in the second frame and the velocity vector.

70. The method of 69, wherein determining a predicted position includes a prediction factor.

71. The method of 67, further comprising:

identifying a second image block in the first frame of the movie;

if the first image block and the second image block are within the search distance of the first trajectory;

determining a first distance between the first image block and the first trajectory;

determining a second distance between the second image block and the first trajectory;

assigning the first image block to the trajectory if the first distance is less than the second distance; and

assigning the second image block to the trajectory if the second distance is less than the first distance.

72. The method of 71, wherein the first distance is determined based on a current position, a predicted position, a velocity, and a predicted velocity of the first image block.

73. The method of 71, wherein the second distance is determined based on a current position, a predicted position, a velocity, and a predicted velocity of the second image block.

74. The method of 67, further comprising:

storing the first trajectory as an unassigned trajectory if no image block in the first frame is within the search distance of the first trajectory.

75. The method of 67, further comprising:

associating one or more characteristics of the first image block to the first trajectory if the first trajectory is assigned to the first image block;

identifying a second image block in the first frame;

assigning a second trajectory to the second image block if the second image block is within a search distance of the second trajectory;

associating one or more characteristics of the second image block to the second trajectory if the second trajectory is assigned to the second image block;

identifying a third image block in a second frame of the movie;

assigning the first and second trajectories to the third image block in the second frame if the third image block is within the search distances of the first and second trajectories,

wherein one or more characteristics of the first image block and the association of the first image block to the first trajectory are stored if the first and second trajectories are assigned to the third image block in the second frame, and

wherein one or more characteristics of the second image block and the association of the second image block to the second trajectory are stored if the first and second trajectories are assigned to the third image block in the second frame; and

associating one or more characteristics of the third image block to the first and second trajectories if the first and second trajectories are assigned to the third image block.

76. The method of 75, wherein the first and second trajectories are assigned to the third image block if the third image block is within a merge distance of the first and second trajectories.

77. The method of 75, further comprising:

identifying a fourth image block in a third frame of the movie; and

assigning the first trajectory or the second trajectory to the fourth image block based on a comparison of one or more characteristics of the first and second image blocks to one or more characteristics of the fourth image block.

78. The method of 77, wherein the first trajectory is assigned to the fourth image block if one or more characteristics of the fourth image block matches one or more characteristics of the first image block more than the second image block.

79. The method of 77, wherein the first trajectory is assigned to the fourth image block if one or more characteristics of the fourth image block and one or more characteristics of the first image block matches within a tolerance.

80. The method of 77, wherein the one or more characteristics include an area.

81. The method of 77, wherein the one or more characteristics include an orientation.

82. The method of 77, wherein the first or second trajectory is assigned to the fourth image block if the fourth image block is within a separation distance of the first and second trajectories.

83. The method of 68, further comprising:

determining a travel distance in a first direction and a second direction based on the velocity vector of the first trajectory.

84. The method of 68, further comprising:

determining a path length based on the velocity vector of the first trajectory.

85. The method of 68, further comprising:

determining a speed based on the velocity vector of the first trajectory.

86. The method of 68, wherein the first trajectory includes a first velocity vector and at least a second velocity vector, and further comprising:

determining an amount of turning based on an angle between the first and second velocity vectors.

87. The method of 68, wherein the second image block includes an orientation, and further comprising:

determining an amount of stumbling based on an angle between the orientation of the second image block and the velocity vector of the first trajectory.

88. A method of processing frames of a digitized movie, the method comprising:

identifying a first image block in a first frame of the movie;

assigning a first trajectory to the first image block;

identifying a second image block in the first frame;

assigning a second trajectory to the second image block;

identifying a third image block in a second frame of the movie,

wherein the first frame precedes the second frame in the movie;

assigning the first and second trajectories to the third image block if the third image block in the second frame is within a specified distance of the first and second trajectories; and

storing one or more characteristics of the first image block in association with the first trajectory and one or more characteristics of the second image block in association with the second trajectory if the third image block is assigned to the first and second trajectories.

89. The method of 88,

wherein first image block is assigned to the first trajectory when the first image block is within a search distance of the first trajectory;

wherein the second image block is assigned to the second trajectory when the second image block is within a search distance of the second trajectory; and

wherein the third image block is assigned to the first and second trajectories when the third image block is within a search distance and a merge distance of the first and second trajectories.

90. The method of 88 further comprising:

identifying a fourth image block in a third frame of the movie,

wherein the second frame precedes the third frame; and

assigning the fourth image block to the first or second trajectory based on a comparison of one or more characteristics of the fourth image block with the one or more stored characteristics associated with the first and second trajectories.

91. The method of 90, wherein the one or more characteristics include an area.

92. The method of 90, wherein the one or more characteristics include an orientation.

93. The method of 90, wherein the one or more characteristics include a velocity.

94. The method of 90, wherein the fourth image block is assigned to the first or second trajectory if the fourth image block is within a separation distance of the first and second trajectories.

95. The method of 88, further comprising:

superimposing frames of the movie to obtain a background approximation; and

determining a characteristic pixel value for pixels of the background approximation based on pixels of the superimposed frames.

96. The method of 95, wherein the characteristic pixel value is an average or a median.

97. The method of 95 further comprising:

subtracting the background approximation from the frames of the movie; and

applying a gray scale threshold to create binary images of the frames.

98. The method of 88,

wherein the first image block includes an orientation;

wherein the first trajectory includes a velocity vector;

wherein an amount of stumbling is determined based on an angle between the orientation of the first image block and the velocity vector of the first trajectory;

wherein the second image block includes an orientation;

wherein the second trajectory includes a velocity vector; and

wherein an amount of stumbling is determined based on an angle between the orientation of the second image block and the velocity vector of the second trajectory.

99. The method of 98, wherein an aggregate amount of stumbling is determined based on the amounts of stumbling determined based on the first image block, the first trajectory, the second image block, and the second trajectory.

100. The method of 98,

wherein the first image block includes a long axis and a short axis;

and

wherein the orientation is determined as an angle between the long axis and a coordinate axis of the first frame.

101. A method of processing frames of a digitized movie, the method comprising:

identifying a first image block in a frame of the movie;

defining a velocity vector for the first image block;

defining an orientation for the first image block; and

determining an amount of stumbling based on an angle between the velocity vector and the orientation.

102. The method of 101, further comprising:

assigning a first trajectory to the first image block;

identifying a second image block in a second frame of the movie,

wherein the first frame precedes the second frame in the movie;

assigning the first trajectory to the second image block if the second image block is within a search distance of the first trajectory; and

determining a velocity vector for the first trajectory based on the position of the first image block in the first frame and the position of the second image block in the second frame.

103. The method of 101, further comprising:

assigning a first trajectory to the first image block;

identifying a second image block in the first frame;

assigning a second trajectory to the second image block;

identifying a third image block in a second frame of the movie,

wherein the first frame precedes the second frame in the movie;

assigning the first and second trajectories to the third image block if the third image block in the second frame is within a merge distance of the first and second trajectories; and

storing one or more characteristics of the first image block in association with the first trajectory and one or more characteristics of the second image block in association with the second trajectory if the third image block is assigned to the first and second trajectories.

104. The method of 103, further comprising:

identifying a fourth image block in a third frame of the movie,

wherein the second frame precedes the third frame; and

assigning the fourth image block to the first or second trajectory based on a comparison of one or more characteristics of the fourth image block with the one or more stored characteristics associated with the first and second trajectories.

105. The method of 104, wherein the one or more characteristics includes one or more of an area, an orientation, and velocity.

106. The method of 104, wherein the fourth image block is assigned to the first or second trajectory if the fourth image block is within a separation distance of the first and second trajectories.

107. The method of 101, further comprising:

superimposing frames of the movie to obtain a background approximation; and

determining a characteristic pixel value for pixels of the background approximation based on pixels of the superimposed frames.

108. The method of 107, wherein the characteristic pixel value is an average or a median.

109. The method of 107, further comprising:

subtracting the background approximation from the frames of the movie; and

applying a gray scale threshold to create binary images of the frames.

110. A system for processing frames of a digitized movie comprising:

a computer storage medium configured to store frames of the movie; and

a processor configured to;

superimpose frames of the movie to obtain a background approximation, and

determine a characteristic pixel value for pixels of the background approximation based on pixels of the superimposed frames.

111. The system of 110, wherein the processor is further configured to:

subtract the background approximation from frames of the movie; and

apply a gray scale threshold to create binary images of the frames.

112. The system of 110, wherein the processor is further configured to:

obtain a first frame from the computer storage medium;

subtract the background approximation from the first frame

apply a gray scale threshold to the first frame;

identify a first image block in the first frame; and

assign a first trajectory to the first image block.

113. The system of 112, wherein the processor is further configured to:

obtain a second frame of the movie from the computer storage medium;

identify a second image block in the second frame;

assign the first trajectory to the second image block if the second image block is within a search distance of the first trajectory; and

determine a velocity vector for the first trajectory based on the position of the first image block in the first frame and the position of the second image block in the second frame.

114. The system of 113, wherein the processor is further configured to:

determine a long axis and a short axis for the second image block;

determine an orientation for the second image block based on an angle between the long axis of the second image block and a coordinate axis of the second frame; and

determine an amount of stumbling based on an angle between the orientation for the second image block and the velocity vector.

115. The system of 112, wherein the processor is further configured to:

obtain a second frame of the movie from the computer storage medium;

identify a second image block in the first frame;

assign a second trajectory to the second image block;

identify a third image block in the second frame;

assign the first and second trajectories to the third image block if the third image block in the second frame is within a merge distance of the first and second trajectories; and

store in the computer storage medium one or more characteristics of the first image block in association with the first trajectory and one or more characteristics of the second image block in association with the second trajectory if the third image block is assigned to the first and second trajectories.

116. The system of 115, wherein the processor is further configured to:

obtain a third frame of the movie from the computer storage medium;

identify a fourth image block in the third frame; and

assign the fourth image block to the first or second trajectory based on a comparison of one or more characteristics of the fourth image block with the one or more stored characteristics associated with the first and second trajectories.

117. The system of 115, wherein the fourth image block is assigned to the first or second trajectory if the fourth image block is within a separation distance of the first and second trajectories.

118. A computer-readable storage medium containing computer executable instructions, the instructions when executed by a computer causing:

superimposing frames of the movie to obtain a background approximation; and

determining a characteristic pixel value for pixels of the background approximation based on pixels of the superimposed frames.

119. The computer-readable storage medium of 118, the instructions when executed further causing:

subtracting the background approximation from a first frame of the movie;

applying a gray scale threshold to the first frame;

identifying a first image block in the first frame; and

assigning a first trajectory to the first image block.

120. The computer-readable storage medium of 119, the instructions when executed further causing:

identifying a second image block in a second frame of the movie;

assigning the first trajectory to the second image block if the second image block is within a search distance of the first trajectory; and

determining a velocity vector for the first trajectory based on the position of the first image block in the first frame and the position of the second image block in the second frame.

121. The computer-readable storage medium of 120, the instructions when executed further causing:

determining a long axis and a short axis for the second image block;

determining an orientation for the second image block based on an angle between the long axis of the second image block and a coordinate axis of the second frame; and

determining an amount of stumbling based on an angle between the orientation for the second image block and the velocity vector.

122. The computer-readable storage medium of 120 the instructions when executed further causing:

identifying a second image block in the first frame;

assigning a second trajectory to the second image block;

identifying a third image block in a second frame of the movie;

assigning the first and second trajectories to the third image block if the third image block in the second frame is within a merge distance of the first and second trajectories; and

storing one or more characteristics of the first image block in association with the first trajectory and one or more characteristics of the second image block in association with the second trajectory if the third image block is assigned to the first and second trajectories.

123. The computer-readable storage medium of 122, the instructions when executed further causing:

identifying a fourth image block in a third frame of the movie; and

assigning the fourth image block to the first or second trajectory based on a comparison of one or more characteristics of the fourth image block with the one or more stored characteristics associated with the first and second trajectories.

124. The computer-readable storage medium of 123, wherein the fourth image block is assigned to the first or second trajectory if the fourth image block is within a separation distance of the first and second trajectories.