Remote contactless stereoscopic mass estimation system

• US 8,351,656 B2
• Filed: 04/29/2010
• Issued: 01/08/2013
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Spicola Tool LLC, Texas Tech University (Texas Tech University System)
• Original Assignee: Spicola Tool LLC, Texas Tech University (Texas Tech University System)
• Inventors: Spicola, Joseph A., Hasanbelliu, Erion, Rubin, Amir C., Sinzinger, Eric, Null, Bradley
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US12/770,268
• Publication Number:
  US 20110196661A1
• Time in Patent Office: 985 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/110, 382/123, 382/154, 382/155, 382/162, 382/168, 382/173, 382/181, 382/189, 382/193, 382/199, 382/203, 382/209, 382/219, 382/232, 382/254, 382/274, 382/276, 382286-294, 382/305, 382/312, 382/321, 452/157, 705/7.25, 356/603, 356/4.03, 380/245
• US Class Current: 382/110
• CPC Class Codes: A01K 29/00 Other apparatus for animal husbandry
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G01G 17/08 for weighing livestock
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume

Abstract

A contactless system and method for estimating the volume, mass or weight of a target animal is provided. First, the target animal is imaged, preferably with a stereoscopic camera. A spatial representation of the target animal is derived from the stereoscopic images. Next, a virtual three-dimensional spatial model is provided, preferably having a shape resembling that of a characteristic of the animal to which the target animal belongs. A software module is provided to reshape the virtual spatial model, using a finite set of independently configurable shape variables, to approximately fit the spatial representation of the individual animal. Finally, the volume, mass or weight of the target animal is estimated as a function of shape variables characterizing the reshaped virtual model.

Claims

1. A system for estimating the volume, mass or weight of an individual animal comprising:

a representation of an individual animal, generated from an image-capturing machine and comprising a plurality of spatially distributed points, stored on a computer readable medium;

a virtual spatial model of a reference object comprising a plurality of spatially distributed points; and

a computer particularly programmed with the capability of reshaping the virtual spatial model, the representation of the individual animal, or both, into an approximate fit with each other and with the capability of matching individual points in the representation of the individual animal with individual points in the virtual spatial model; and

a camera for capturing the representation of a visible or spatial characteristic of the individual animal, such that the camera and the computer are included in a hand-held unit,

wherein the computer is particularly programmed to estimate the mass, weight, or volume of the individual animal as a function of an amount that the virtual spatial model, the representation of the individual animal, or both, were reshaped.

2. The system of claim 1, further characterized in that the computer is particularly programmed with the automatic capability of reshaping the virtual spatial model, the representation of the individual animal, or both, into an approximate fit with each other.

3. The system of claim 1, further characterized in that the computer is particularly programmed with the automatic capability of rotating the representation of the individual animal, the virtual spatial model, or both, so that the representation of the individual animal and the virtual spatial model have substantially similar orientations.

4. The system of claim 1, further characterized in that the computer is particularly programmed with the capability of automatically cropping a portion of an original image that surrounds a representation of the individual animal in order to generate the representation of the individual animal.

5. The system of claim 1, further characterized in that the computer is particularly programmed with the capability of automatically partitioning both the representation of an individual animal and the virtual spatial model in order to facilitate the reshaping of the virtual spatial model, the representation of the individual animal, or both, into an approximate fit with each other.

6. The system of any of claim 1, further characterized in that the virtual spatial model is a multi-dimensional profile of a reference animal.

7. The system of claim 6, further characterized in that the virtual spatial model is a cow model.

8. The system of any of claim 7, wherein the individual animal is a mammal.

9. The system of claim 8, wherein the mammal is a livestock animal.

10. A system for estimating the volume, mass or weight of an individual animal comprising:

a representation of a visible or spatial characteristic of the individual animal stored on a computer readable medium;

a spatially multi-dimensional virtual model that is shape-configurable through a set of at least two independent shape transformation variables, stored on a computer readable medium, of an object; and

a computer particularly programmed with the capability of

transforming the virtual model, the representation of the individual animal, or both, into an optimal fit with each other, using at least two independent transformation parameters;

selecting values for the shape transformation variables to reshape the virtual model into an optimal fit with the representation of the individual animal;

estimating the volume, mass or weight of the individual animal as a function of the selected values for the shape transformation variables used to reshape the virtual model; and

estimating the mass or weight of the individual animal as a function of a plurality of independent relational parameters that relate pre- and post-transformed versions of the virtual model or the representation of the individual animal.

11. The system of claim 10, wherein the computer is particularly programmed with the capability of automatically identifying suitable values for variables of a volume-estimating, mass-estimating, or weight-estimating function, wherein the variables of the function are the same as or derived from the transformation parameters used to approximately fit the representation of the individual animal and the virtual model with each other.

12. The system of claim 10, wherein the computer is particularly programmed with the capability of automatically identifying suitable values for variables of a volume-estimating, mass-estimating, or weight-estimating function, wherein the variables of the function are the same as or derived from the relational parameters that relate the pre- and post-transformed versions of the representation of the individual animal or the virtual model.

13. The system of claim 10, further characterized in that:

the virtual model is shape-configurable through a set of at least two independent shape transformation variables;

the computer is particularly programmed with the capability of selecting values for the shape transformation variables to reshape the virtual model into an optimal fit with the representation of the individual animal;

the computer is particularly programmed with the capability, after the virtual model has been reshaped into an optimal fit with the representation of the individual animal, of determining a set of relational parameters that relate a pre-transformed version of the virtual model with a post-transformed version of the virtual model; and

the computer is particularly programmed with the capability of estimating the volume, mass or weight of the individual animal as a function of the relational parameters.

14. The system of claim 10, wherein the computer is particularly programmed with the capability of aligning the representation of the animal with the virtual model so that the representation of the animal and the virtual model have substantially similar orientations.

15. The system of claim 14, wherein the computer is particularly programmed with the capability of rotating and translating the virtual model to match the orientation of the representation of the animal.

16. The system of claim 14, wherein the computer is particularly programmed with the capability of rotating and translating the representation of the animal to match the orientation of the virtual model.

17. The system of claim 10, wherein the computer is particularly programmed with the capability of at least one of the set consisting of:

automatically rotating and translating the virtual model into an orientation that approximately matches an orientation of the representation of the individual animal; and

automatically rotating and translating the representation of the individual animal into an orientation that approximately matches an orientation of the virtual model.

18. The system of claim 17, wherein the computer is particularly programmed to transform the virtual model to approximately fit the representation of the individual animal using at least two independently determined shape transformation variables.

19. The system of claim 10, further comprising a camera, communicatively coupled to the computer, that generates a representation of a visible or spatial characteristic of the individual animal.

20. The system of claim 10, further comprising a stereo camera, communicatively coupled to the computer, that generates a representation of a visible or spatial characteristic of the individual animal.

21. The system of claim 20, wherein the stereo camera is operable to capture a stereoscopic image of the individual animal and the computer is particularly programmed to transform the stereoscopic image into a three-dimensional representation of the animal, wherein the representation of a visible or spatial characteristic of the individual animal consists of said three-dimensional representation.

22. The method of claim 20, further comprising automatically registering the point cloud with the virtual model.

23. The method of claim 22, further comprising transforming the point cloud and virtual model to fit each other using a matched filter routine.

24. The method of claim 22, further comprising cropping the point cloud and then matching each point in the cropped point cloud with a corresponding point in the virtual model.

25. The method of claim 24, further comprising matching each point in the cropped point cloud with a nearest point of the virtual model.

26. The method of claim 25, further comprising portioning both the cropped point cloud and the virtual model into a plurality of partitions corresponding to different parts of the animal.

27. The method of claim 26, wherein one of the partitions corresponds substantially only to one of the following parts of a cow:

the head;

the front legs;

the rear legs;

the dewlap; and

the sheath.

28. The method of claim 26, wherein each matched point to a point in a partition of the cropped point cloud is the nearest point in a corresponding partition of the virtual model.

29. The system of claim 26, further comprising assigning partition-specific weights to points in different partitions of the cropped point cloud, and transforming the cropped point cloud and virtual model to fit each other by an amount that is a function of the partition-specific weights.

30. The method of claim 25, further comprising assigning a weight to each point in the cropped point cloud that is a function of the spatial distance between that point and its matching virtual model point.

31. The system of claim 10, wherein the animal is a human.

32. The system of claim 10, wherein the representation comprises a stereo image representation of the animal.

33. The system of claim 10, wherein the representation comprises a three-dimensional representation of the animal.

34. The system of claim 10, wherein the representation comprises a projection of the animal on a plane.

35. The system of claim 34, wherein the representation includes depth data corresponding to an estimated distance from the plane to each point of the animal being projected onto the plane.

36. The method of claim 34, further comprising automatically removing points in the cropped point cloud linked to the blob origin point cloud through removed ground points.

37. The system of claim 10, further comprising a machine that captures a representation of a scene including the individual animal, and wherein the computer automatically crops a portion of the scene surrounding the individual animal.

38. The system of claim 37, wherein the machine is programmed to automatically distinguish the animal from the surrounding scene.

39. The system of claim 10, wherein the representation comprises a non-color-specific three-dimensional profile of the animal.

40. The system of claim 10, further comprising a machine that captures and stores the representation in the form of a point cloud having spatial coordinates.

41. The system of claim 10, wherein the representation of a visible or spatial characteristic of the individual animal comprises a point cloud having spatial coordinates.

42. The system of claim 41, wherein the point cloud represents a projection of the individual animal on an image plane coupled with depth data, the depth data providing the estimated relative depth—

from the perspective of the camera—

of each point in the representation.

43. The system of claim 41, wherein the computer is programmed the capability of automatically isolating a group of spatially proximate points presumed to include the individual animal from the point cloud.

44. The system of claim 41, wherein the points at the center of the point cloud, hereinafter referred to as blob origin points, are presumed to represent a portion of the target animal;

and

wherein the computer is programmed with the capability of automatically identifying points in the cloud that are spatially proximate to the blob origin points and all other points in the cloud that are connected to the blob origin points through intermediate spatially proximate points.

45. The system of claim 44, wherein the computer is programmed with a ground-removal routine capable of automatically cropping the point cloud to exclude points presumed to represent a ground surface on which the individual animal was standing.

46. The system of claim 45, wherein the computer is programmed with the capability of automatically isolating points in the point cloud presumed to represent a portion of the target animal'"'"'s legs in furtherance of the ground-removal routine.

47. The system of claim 46, wherein the computer is programmed with the capability of automatically removing points in the cropped point cloud linked to the blob origin point cloud through removed ground points.

48. The system of claim 41, wherein the point cloud is cropped and then each point in the cropped point cloud is matched with a corresponding point in the virtual model.

49. The system of claim 48, wherein each point in the cropped point cloud is matched with a nearest point of the virtual model.

50. The system of claim 48, wherein each point in the cropped point cloud is assigned a weight that is a function of the spatial distance between that point and its matching virtual model point.

51. The system of claim 41, wherein the computer is programmed to iteratively transform the point cloud and virtual model into an increasingly optimal fit.

52. The system of claim 51, wherein during each iteration a point in the point cloud is matched with a nearest corresponding point in the virtual model.

53. The system of claim 52, wherein during each iteration the point cloud image is translated as a function of the weighted sum of the distances between each point in the point cloud and its nearest corresponding point in the virtual model.

54. The system of claim 53, wherein during each iteration the point cloud image is also rotated as a function of weight values assigned to each point in the point cloud.

55. The system of claim 54, wherein during each iteration the virtual model is also scaled to increasingly fit the point cloud.

56. A method for estimating the volume, mass or weight of an individual animal comprising:

machine generating a representation of a visible or spatial characteristic of the individual animal comprising a point cloud having spatial coordinates such that points at the center of the point cloud, hereinafter referred to as blob origin points, are presumed to represent a portion of the target animal;

automatically identifying points in the cloud that are spatially proximate to the blob origin points and all other points in the cloud that are connected to the blob origin points through intermediate spatially proximate points;

obtaining a virtual model, stored on a computer readable medium, of a spatially multidimensional object;

transforming, with the aid of a computer, the virtual model, the representation of the individual animal, or both, into an optimal fit with each other, using at least two independent transformation parameters; and

estimating, with the aid of a computer, the volume, mass or weight of the individual animal as a function of a plurality of independent relational parameters that relate pre- and post-transformed versions of the virtual model or the representation of the individual animal.

57. The method of claim 56, wherein the virtual model is a multi-dimensional profile of a reference animal.

58. The method of claim 56, further comprising automatically identifying suitable values for variables of a volume-estimating, mass-estimating, or weight-estimating function, wherein the variables of the volume-estimating, mass-estimating, or weight-estimating function are the same as or derived from the transformation parameters used to approximately fit the representation of the individual animal and the virtual model with each other.

59. The method of claim 56, further comprising automatically identifying suitable values for variables of a volume-estimating, mass-estimating, or weight-estimating function, wherein the variables of the volume-estimating, mass-estimating, or weight-estimating function are the same as or derived from the relational parameters that relate the pre- and post-transformed versions of the representation of the individual animal or the virtual model.

60. The method of claim 56, further characterized in that the virtual model is shape-configurable through a set of at least two independent shape transformation variables, the method further comprising:

selecting values for the shape transformation variables to reshape the virtual model into an optimal fit with the representation of the individual animal; and

estimating the volume, mass or weight of the individual animal as a function of the selected values for the shape transformation variables used to reshape the virtual model.

61. The method of claim 56, further characterized in that the virtual model is shape-configurable through a set of at least two independent shape transformation variables, the method further comprising:

selecting values for the shape transformation variables to reshape the virtual model into an optimal fit with the representation of the individual animal;

after the virtual model has been reshaped into an optimal fit with the representation of the individual animal, determining a set of relational parameters that relate a pre-transformed version of the virtual model with a post-transformed version of the virtual model; and

estimating the volume, mass or weight of the individual animal as a function of the relational parameters.

62. The method of claim 56, further comprising aligning the representation of the animal with the virtual model so that the representation of the animal and the virtual model have substantially similar orientations.

63. The method of claim 62, further comprising rotating and translating the virtual model to match the orientation of the representation of the animal.

64. The method of claim 62, further comprising rotating and translating the representation of the animal to match the orientation of the virtual model.

65. The method of claim 62, wherein the representation includes depth data corresponding to an estimated distance from the plane to each point of the animal being projected onto the plane.

66. The method of claim 56, further comprising automatically rotating and translating the virtual model into an orientation that approximately matches an orientation of the representation of the individual animal.

67. The method of claim 56, further comprising automatically rotating and translating the representation of the individual animal into an orientation that approximately matches an orientation of the virtual model.

68. The method of claim 67, further comprising transforming the virtual model to approximately fit the representation of the individual animal using at least two independently determined shape transformation variables.

69. The method of claim 56, wherein the animal is a human.

70. The method of claim 56, wherein the representation comprises a stereo image representation of the animal.

71. The method of claim 56, wherein the representation comprises a three-dimensional representation of the animal.

72. The method of claim 56, wherein the representation comprises a projection of the animal on a plane.

73. The method of claim 56, further comprising:

capturing a representation of a scene including the individual animal; and

automatically cropping a portion of the scene surrounding the individual animal.

74. The method of claim 73, further comprising automatically, and with the aid of a computer, distinguishing the animal from the surrounding scene.

75. The method of claim 56, wherein the representation comprises a non-color-specific three-dimensional profile of the animal.

76. The method of claim 75, further comprising automatically identifying a left-or-right orientation of the animal represented in the point cloud.

77. The method of claim 56, further comprising storing the representation in the form of a point cloud having spatial coordinates.

78. The method of claim 56, wherein the point cloud represents a projection of the individual animal on an image plane coupled with depth data, the depth data providing the estimated relative depth—

from the perspective of the camera—

of each point in the representation.

79. The method of claim 56, further comprising automatically isolating a group of spatially proximate points presumed to include the individual animal from the point cloud.

80. The method of claim 56, further comprising automatically cropping the point cloud to exclude points presumed to represent a ground surface on which the individual animal was standing.

81. The method of claim 80, further comprising automatically isolating points in the point cloud presumed to represent a portion of the target animal'"'"'s legs in furtherance of the ground-removal routine.

82. The method of claim 56, wherein the virtual model comprises a computer-aided-design model of the reference animal.

83. The method of claim 56, wherein the virtual model comprises a set of spatially-determined points, each point having a spatial coordinate.

84. The method of claim 56, further comprising iteratively transforming the point cloud and virtual model into an increasingly optimal fit.

85. The method of claim 84, further comprising, during each iteration, matching a point in the point cloud with a nearest corresponding point in the virtual model.

86. The method of claim 85, wherein during each iteration, translating the point cloud image as a function of the weighted sum of the distances between each point in the point cloud and its nearest corresponding point in the virtual model.

87. The method of claim 86, wherein during each iteration, rotating the point cloud image as a function of weight values assigned to each point in the point cloud.

88. The method of claim 87, wherein during each iteration, scaling the virtual model to increasingly fit the point cloud.

89. A system for estimating the volume, mass or weight of an individual animal comprising:

a machine that generates a representation of a visible or spatial characteristic of the individual animal;

a computer communicatively coupled to the machine; and

a spatially multi-dimensional virtual model, stored on a computer readable medium, of an object;

wherein the computer is particularly programmed with the capability of transforming the virtual model, the representation of the individual animal, or both, into an optimal fit with each other, using at least two independent transformation parameters;

wherein the computer is particularly programmed to estimate the mass or weight of the individual animal as a function of a plurality of independent relational parameters that relate pre- and post-transformed versions of the virtual model or the representation of the individual animal; and

wherein the computer is particularly programmed with the capability of automatically identifying suitable values for variables of a volume-estimating, mass-estimating, or weight-estimating function, wherein the variables of the function are the same as or derived from the relational parameters that relate the pre- and post-transformed versions of the representation of the individual animal or the virtual model.

90. The system of claim 89, wherein the virtual model is a multi-dimensional profile of a reference animal.

91. The system of claim 89, wherein the computer is particularly programmed with the capability of automatically identifying suitable values for variables of a volume-estimating, mass-estimating, or weight-estimating function, wherein the variables of the function are the same as or derived from the transformation parameters used to approximately fit the representation of the individual animal and the virtual model with each other.

92. The system of claim 89, wherein the machine and computer are packaged within a mobile, portable, hand-held unit.

93. A system for estimating the volume, mass or weight of an individual animal comprising:

a machine that generates a representation of a visible or spatial characteristic of the individual animal;

a computer communicatively coupled to the machine; and

a spatially multi-dimensional virtual model, stored on a computer readable medium, of an object such that the virtual model of the reference animal is shape-configurable through a set of at least two independent shape transformation variables;

the computer is particularly programmed with the capability of selecting values for the shape transformation variables to reshape the virtual model into an optimal fit with the representation of the individual animal;

the computer is particularly programmed to estimate the volume, mass or weight of the individual animal as a function of the selected values for the shape transformation variables used to reshape the virtual model

the computer is particularly programmed with the capability of transforming the virtual model, the representation of the individual animal, or both, into an optimal fit with each other, using at least two independent transformation parameters; and

the computer is particularly programmed to estimate the mass or weight of the individual animal as a function of a plurality of independent relational parameters that relate pre- and post-transformed versions of the virtual model or the representation of the individual animal.