System and method to determine a teat position using edge detection in rear images of a livestock from two cameras

• US 9,681,634 B2
• Filed: 04/17/2012
• Issued: 06/20/2017
• Est. Priority Date: 04/28/2011
• Status: Active Grant

• Current Assignee: Technologies Holdings Corporation (Teleflex Incorporated)
• Original Assignee: Technologies Holdings Corporation (Teleflex Incorporated)
• Inventors: Hofman, Henk, de Ruijter, Cor, Koekoek, Menno, van der Sluis, Peter Willem
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): SUH, JOSEPH JINWOO
• Application Number:
  US13/448,751
• Publication Number:
  US 20120274761A1
• Time in Patent Office: 1,890 Days
• Field of Search: 348135
• CPC Class Codes: A01J 5/007 Monitoring milking processes; Control or regulation of milking machines
• CPC Class Codes: A01J 5/0175 Attaching of clusters
• CPC Class Codes: G06T 2207/30128 Food products
• CPC Class Codes: G06T 7/75 involving models

Abstract

A system includes a first camera, a second camera, and a processor wherein the second camera has a higher resolution than the first camera. The processor is communicatively coupled to the first camera and the second camera and is operable to determine a center coordinate of an udder of a dairy livestock based at least in part upon visual data captured by the first camera. The processor is also operable to determine a position of a teat of the dairy livestock based at least in part upon the center coordinate and visual data captured by the second camera.

Claims

1. A system, comprising:

a first camera operable to capture an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values;

a second camera, the second camera having a higher resolution than the first camera; and

a processor communicatively coupled to the first camera and the second camera, the processor operable to;

determine edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;

determine a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock; and

determine a position of a teat of the dairy livestock based at least in part upon the center coordinate and visual data captured by the second camera.

2. The system of claim 1, wherein the position of the teat is relative to the center coordinate.

3. The system of claim 1, further comprising a memory communicatively coupled to the processor, the memory operable to store a plurality of coordinates comprising an x-coordinate, a y-coordinate, and a z-coordinate for each teat of the dairy livestock, wherein each coordinate is in relation to the center of the udder of the dairy livestock.

4. The system of claim 1, wherein the first camera is a three-dimensional camera and the second camera is a two-dimensional camera.

5. The system of claim 1, further comprising a tracker operable to track the rear of the dairy livestock relative to the rear of a milking stall.

6. The system of claim 5, further comprising an encoder communicatively coupled to the tracker and operable to determine a distance that the tracker moves.

7. A method, comprising:

capturing an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values;

determining edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;

determining a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock; and

determining a position of a teat of the dairy livestock based at least in part upon the center coordinate and visual data captured by a second camera, wherein the second camera has a higher resolution than the first camera.

8. The method of claim 7, wherein the position of the teat is relative to the center coordinate.

9. The method of claim 7, further comprising storing a plurality of coordinates comprising an x-coordinate, a y-coordinate, and a z-coordinate for each teat of the dairy livestock, wherein each coordinate is in relation to the center of the udder of the dairy livestock.

10. The method of claim 7, wherein the first camera is a three-dimensional camera and the second camera is a two-dimensional camera.

11. The method of claim 7, further comprising determining the position of the rear of the dairy livestock relative to the rear of a milking stall.

12. A system, comprising:

a memory operable to store an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values; and

a processor communicatively coupled to the memory, the processor operable to;

determine edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;

determine a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock;

determine a z-coordinate of a reference point of a teat of the diary livestock based at least in part upon stored coordinates of a dairy livestock and the determined center coordinate of the udder of the dairy livestock;

determine an x-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock; and

determine a y-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock.

13. The system of claim 12, further comprising a two-dimensional camera comprising a laser device, wherein the two-dimensional camera is communicatively coupled to the processor.

14. The system of claim 13, further comprising a robotic arm, wherein the two-dimensional camera is attached to the robotic arm.

15. The system of claim 14, wherein the processor is further operable to control the robotic arm to move to the reference point.

16. The system of claim 13, wherein the processor is further operable to:

determine a z-coordinate of the teat based at least in part upon visual data captured by the two-dimensional camera;

determine a y-coordinate of the teat based at least in part upon the visual data captured by the two-dimensional camera; and

determine an x-coordinate of the teat based at least in part upon the y-coordinate of the teat and the visual data captured by the two-dimensional camera.

17. The system of claim 16, wherein determining the y-coordinate comprises determining the tip of the teat.

18. The system of claim 16, wherein the visual data captured by the two-dimensional camera comprises a reflection of a laser emitted by the laser device.

19. The system of claim 18, wherein determining the x-coordinate comprises:

determining an angle based at least in part upon the laser; and

determining a distance of the teat from the camera based at least in part upon the angle and the y-coordinate of the teat.

20. The system of claim 12, further comprising a tracker operable to track the rear of the dairy livestock relative to the rear of a milking stall.

21. The system of claim 20, wherein the displacement measurement is based at least in part on the tracker.

22. A method comprising:

storing an image of a rear of a dairy livestock, the image comprising a plurality of pixels having depth values;

determining edges of the rear of the dairy livestock based at least in part upon the depth values of the plurality of pixels;

determining a center coordinate of an udder of the dairy livestock based at least in part upon the determined edges of the rear of the dairy livestock;

determining a z-coordinate of a reference point, using a processor, of a teat of the diary livestock based at least in part upon stored coordinates of a dairy livestock and the determined center coordinate of the udder of the dairy livestock;

determining an x-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock; and

determining a y-coordinate of the reference point based at least in part upon the stored coordinates and the determined center coordinate of the dairy livestock.

23. The method of claim 22, wherein the processor is communicatively coupled to a two-dimensional camera, the two-dimensional camera comprising a laser device.

24. The method of claim 23, wherein the two-dimensional camera is coupled to a robotic arm.

25. The method of claim 24, further comprising controlling a robotic arm to move to the reference point.

26. The method of claim 23, further comprising:

determining a z-coordinate of the teat based at least in part upon visual data captured by the two-dimensional camera;

determining a y-coordinate of the teat based at least in part upon the visual data captured by the two-dimensional camera; and

determining an x-coordinate of the teat based at least in part upon the z-coordinate of the teat and the visual data captured by the two-dimensional camera.

27. The method of claim 26, wherein determining the y-coordinate comprises determining the tip of the teat.

28. The system of claim 26, wherein the visual data captured by the two-dimensional camera comprises a reflection of a laser emitted by the laser device.

29. The method of claim 28, wherein determining the x-coordinate comprises:

determining an angle based at least in part upon the laser; and

determining a distance of the teat from the camera based at least in part upon the angle and the y-coordinate of the teat.

30. The method of claim 22, further comprising tracking, using a tracker, the rear of the dairy livestock relative to the rear of a milking stall.