AGRICULTURAL MONITORING SYSTEM USING IMAGE ANALYSIS

• US 20160353715A1
• Filed: 06/03/2016
• Published: 12/08/2016
• Est. Priority Date: 06/03/2015
• Status: Active Grant

• Current Assignee: Keltronix, Inc.
• Original Assignee: Keltronix, Inc.
• Inventors: Simpson, Jonathan, Temby, Kelton
• Granted Patent: US 9,992,977 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A01K 47/06 Other details of beehives, e.g. ventilating devices, entrances to hives, guards, partitions or bee escapes
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion

Abstract

Aspects of this disclosure include a system for providing non-contact, computer-vision based monitoring of the health and pollination activity of a beehive. The system may include camera positioned proximate to a beehive. The camera may include an onboard processor that analyzes video of the beehive captured by the camera and calculates an activity value that estimates a number of bees moving about the beehive. The video calculated activity values may be uploaded to a server where they can be accessed via a user device. The user device may allow the user to display interactive plots of the activity values over a variety of time bases. The disclosed beehive monitoring system relies on relatively lost-cost hardware and requires neither modification to the hive nor special constraints on the placement of the camera.

Claims

1. An apparatus for monitoring a beehive, the apparatus comprising:

a camera that captures a sequence of video frames of the exterior of the beehive at a predetermined frame rate, the sequence of video frames including at least a first video frame, a second video frame, and a third video frame; and

,

a processor in communication with the camera system, the processor performs image analysis on the sequence of video frames to calculate an activity value that represents an estimate of the number of bees in the vicinity of the beehive, wherein the image analysis performed by the processor includes;

generating a first difference frame by subtracting the second video frame from the first video frame;

generating a second difference frame by subtracting the third video frame from the second video frame;

generating a differential image frame by combining the first difference frame and the second difference frame using a logical AND operation;

generating a modified differential image frame by performing a thresholding operation on the differential image frame;

identifying a set of contours in the modified differential image frame by performing blob detection on the modified differential image frame;

generating a count of the number of contours in the set of contours having an area that falls within a predefined range;

calculating the activity value by multiplying the count by the predetermined frame rate; and

,

storing the activity value on a storage device for display on a user device.

2. The apparatus of claim 1 further comprising a local storage device for storing the activity value in association with the sequence of video frames.

3. The apparatus of claim 1 further comprising a communication system in communication with the processor, wherein the communication system transmits the activity value to a server for retrieval by the user device.

4. The apparatus of claim 3, wherein the communication system transmits the sequence of video frames to the server.

5. The apparatus of claim 3, wherein the communication system transmits data received from one or more environmental sensors to the server.

6. The apparatus of claim 1 further comprising a mobile platform coupled to the camera system.

7. The apparatus of claim 1 further comprising solar-powered power supply and a battery.

13. The apparatus of claim 7 further comprising a mobile platform coupled to the camera system.

14. The apparatus of claim 7 further comprising a solar-powered power supply and a battery.

8. An apparatus for monitoring a beehive, the apparatus comprising:

a camera that captures a sequence of video frames of the exterior of a beehive; and

,

a processor in communication with the camera system, the processor performs image analysis on the sequence of video frames to calculate an activity value that represents an estimate of the number of bees in the vicinity of the beehive, wherein the image analysis performed by the processor includes;

generating a series of contour count values representing a number of contours identified in each frame, wherein generating the series includes;

generating an average frame by combining a current frame and at least one previous frame;

generating a differential frame by subtracting the average frame from the current frame;

generating a first modified differential frame by performing an erosion operation on the differential frame;

generating a second modified differential frame by performing a dilation operation on the first modified differential frame;

identifying a set of contours in the second modified differential frame; and

,

generating a count of the number of contours in the set of contours;

generating a filtered series of contour count values by applying a low-pass filter to the series of contour count values;

incrementing the activity value for each increase in the filtered series of contour values; and

,

storing the activity value on a storage device for display on a user device.

9. The apparatus of claim 8 further comprising a local storage device for storing the activity value in association with the sequence of video frames.

10. The apparatus of claim 8 further comprising a communication system in communication with the processor, wherein the communication system transmits the activity value to a server for retrieval by the user device.

11. The apparatus of claim 9, wherein the communication system transmits the sequence of video frames to the server.

12. The apparatus of claim 9, wherein the communication system transmits data received from one or more environmental sensors to the server.

15. A system for monitoring the health of a beehive, the system comprising:

a camera that captures a sequence of video frames of the exterior of the beehive;

a processor in communication with the camera, the processor analyzes the sequence of video frames to calculate one or more activity values that represents an estimate of the number of bees in the vicinity of the beehive in the sequence of video frames;

a server in communication with the processor, the server stores the sequence of video frames in association with the one or more activity values; and

,

a user device in communication with the server, the user device displays a plot of activity values over a time base.

16. The system of claim 15, wherein the plot displayed on the user device is interactive and allows the user to modify the time base of the plot.

17. The system of claim 15, wherein the plot is interactive and allows the user to select a time index and play back a video sequence associated with the selected time index.

18. The system of claim 15, wherein the plot of activity values include peak activity values.

19. The system of claim 15, wherein the plot of activity values include average activity values.

20. The system of claim 15, wherein the server obtains first and second activity value series for a predetermined time period from first and second beehives, respectively, determines a correlation value based on the first and second activity value series, and transmits a notification to the user device if the correlation value falls below a predetermined threshold.