Methods for agronomic and agricultural monitoring using unmanned aerial systems

• US 9,922,405 B2
• Filed: 08/20/2015
• Issued: 03/20/2018
• Est. Priority Date: 08/22/2014
• Status: Active Grant

• Current Assignee: Climate LLC (Bayer AG)
• Original Assignee: The Climate Corporation (Bayer AG)
• Inventors: Sauder, Doug, Koch, Justin L., Plattner, Troy L., Baurer, Phil
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Rivera-Martinez, Guillermo
• Application Number:
  US14/831,165
• Publication Number:
  US 20160050840A1
• Time in Patent Office: 943 Days
• CPC Class Codes: A01B 79/005 Precision agriculture
• CPC Class Codes: B64U 20/87 Mounting of imaging devices, e.g. mounting of gimbals
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/40 for agriculture or forestry operations
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/106 Change initiated in response to external conditions, e.g. avoidance of elevated terrain or of no-fly zones
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 20/188 Vegetation

Abstract

A method for agronomic and agricultural monitoring includes designating an area for imaging, determining a flight path above the designated area, operating an unmanned aerial vehicle (UAV) along the flight path, acquiring images of the area using a camera system attached to the UAV, and processing the acquired images.

Claims

1. A method for agronomic and agricultural monitoring, the method comprising:

monitoring, by a first central processing unit (CPU) at a base station, an unmanned aerial vehicle (UAV), as the UAV flies along a flight path above an area and as the UAV performs;

capturing, by a second CPU of the UAV, a plurality of initial images of the area as the UAV flies along the flight path;

analyzing, by the second CPU of the UAV, the plurality of initial images of the area to determine whether one or more images, of the plurality of initial images, depict one or more target areas having nitrogen levels below a certain threshold;

in response to determining that the one or more images, of the plurality of initial images, depict the one or more target areas having the nitrogen levels below the certain threshold, determining an identification of the one or more target areas for taking one or more additional different images at a high resolution;

in response to receiving the identification, causing, by the second CPU of the UAV, the UAV to capture the one or more additional different images of the one or more target areas;

transmitting the plurality of initial images and the one or more additional different images to an image recipient.

2. The method of claim 1, further comprising the UAV identifying itself the one or more target areas for which the one or more different images are required as the UAV orthorectifies the plurality of initial images and identifies areas with a low quality imagery.

3. The method of claim 1, further comprising:

sending, by the UAV, to a computing device an indication of areas having certain characteristics comprising one or more of low nitrogen levels, low crop maturity levels, or high crop shadow levels;

receiving, from the computing device, instructions to acquire one or more images of the one or more target areas for which the one or more additional different images are required;

receiving, from the computing device, an identification of the one or more target areas for which the one or more additional different images are required.

4. The method of claim 1, further comprising, capturing, by the UAV, the one or more additional different images at a higher resolution than a resolution at which the plurality of initial images was captured.

5. The method of claim 1, further comprising, capturing, by the UAV, the one or more different images at a lower elevation.

6. The method of claim 1, further comprising transferring the plurality of initial images and the one or more additional different images to the image recipient as the images are captured by the UAV and while the UAV is airborne.

7. The method of claim 1, further comprising determining the flight path by:

receiving, at the base station, an input that indicates a type of image to be acquired;

receiving, at the base station, obstacle data indicating an obstacle within the area; and

determining, by the base station, the flight path based on at least in part the input and the obstacle data.

8. The method of claim 1, further comprising, processing, by the UAV, the plurality of initial images and the one or more additional different images on-board as the UAV flies over the area.

9. The method of claim 1, further comprising, processing, by the UAV, the plurality of initial images and the one or more additional different images by orthorectifying and stitching the plurality of initial images and the one or more additional different images into a single continuous area map.

10. The method of claim 1, further comprising:

processing, by the base station or a cloud computer, the plurality of initial images and the one or more additional different images by performing one or more of;

orthorectifying and stitching the plurality of initial images and the one or more different images into a single continuous area map;

superimposing the plurality of initial images and the one or more different images over other types aerial geographic images;

displaying the plurality of initial images and the one or more additional different images on a graphical user interface for a user;

based on the plurality of initial images and the one or more additional different images, generating a visual animation and displaying the visual animation on the graphical user interface for the user;

filtering the plurality of initial images and the one or more additional different images by applying one or more filters to the plurality of initial images and the one or more additional different images;

based on the plurality of initial images and the one or more additional different images, determining a greenness density map for the area, wherein the greenness density map has an expected greenness density area depicted in a first color and other areas depicted in a second color;

based on the plurality of initial images and the one or more additional different images, generating at least one of a normalized difference vegetation index map, or an application map for the area;

based on the plurality of initial images and the one or more additional different images, determining an emergence map for the area;

based on the plurality of initial images and the one or more additional different images, generating at least one of a normalized difference vegetation index map, or an application map;

based on the plurality of initial images and the one or more additional different images, generating one or more agronomic recommendations for the area;

wherein the processing of the plurality of initial images and the one or more different images is performed at one or more of;

the base station, or a cloud system.

11. The method of claim 10, further comprising processing the emergence map for the area by:

identifying rows of areas above an expected greenness density on the emergence map;

partitioning areas above the expected greenness density into individual areas each corresponding to one plant of a plurality of plants and consistent with planting data;

identifying structures of at least one plant of the plurality of plants;

calculating an estimated yield from at least one plant of the plurality of plants based on the identified structures;

generating a report including the estimated yield.

12. A system for agronomic and agricultural monitoring, the system comprising:

a base station comprising a first central processing unit (CPU) and configured to monitor one or more unmanned aerial vehicles as the one or more unmanned aerial vehicles fly along a flight path above an area;

an unmanned aerial vehicle (UAV) comprising a second CPU and configured to;

the second CPU of the UAV, capturing a plurality of initial images of the area as the UAV flies along the flight path;

the second CPU of the UAV, analyzing the plurality of initial images of the area to determine whether one or more images, of the plurality of initial images, depict one or more target areas having nitrogen levels below a certain threshold;

in response to determining that the one or more images, of the plurality of initial images, depict the one or more target areas having the nitrogen levels below the certain threshold, determining an identification of the one or more target areas for taking one or more additional different images at a high resolution;

the second CPU of the UAV, in response to receiving the identification;

causing the UAV to capture the one or more additional different images of the one or more target areas;

transmitting the plurality of initial images and the one or more additional different images to an image recipient.

13. The system of claim 12, wherein the UAV is further configured to identify itself the one or more target areas for which the one or more additional different images are required as the UAV orthorectifies the plurality of initial images and identifies areas with a low quality imagery.

14. The system of claim 12, wherein the UAV is further configured to:

send to a computing device an indication of areas having certain characteristics comprising one or more of low nitrogen levels, low crop maturity levels, or high crop shadow levels;

receive, from the computing device, instructions to acquire one or more images of the one or more target areas for which the one or more additional different images are required;

receive, from the computing device, an identification of the one or more target areas for which the one or more additional different images are required.

15. The system of claim 12, wherein the UAV is further configured to:

capture the one or more additional different images at a higher resolution than a resolution at which the plurality of initial images was captured.

16. The system of claim 12, wherein the UAV is further configured to:

capture the one or more different images at a lower elevation.

17. The system of claim 12, wherein the UAV is further configured to:

transfer the plurality of initial images and the one or more additional different images to the image recipient as the images are captured by the UAV and while the UAV is airborne.

18. The system of claim 12, wherein the base station is further configured to:

receive a user input that indicates a type of image to be acquired;

receive obstacle data indicating an obstacle within the area; and

determine the flight path based on at least in part the user input and the obstacle data.

19. The system of claim 12, wherein the UAV is further configured to:

process the plurality of initial images and the one or more additional different images on-board as the UAV flies over the area.

20. The system of claim 12, wherein the UAV is further configured to process the plurality of initial images and the one or more additional different images by orthorectifying and stitching the plurality of initial images and the one or more additional different images into a single continuous area map.

21. The system of claim 12, wherein a cloud computer processes the plurality of initial images and the one or more additional different images by performing one or more of:

orthorectifying and stitching the plurality of initial images and the one or more additional different images into a single continuous area map;

superimposing the plurality of initial images and the one or more additional different images over other types aerial geographic images;

displaying the plurality of initial images and the one or more additional different images on a graphical user interface for a user;

based on the plurality of initial images and the one or more additional different images, generating a visual animation and displaying the visual animation on the graphical user interface for the user;

filtering the plurality of initial images and the one or more additional different images by applying one or more filters to the plurality of initial images and the one or more additional different images;

based on the plurality of initial images and the one or more additional different images, determining a greenness density map for the area, wherein the greenness density map has an expected greenness density area depicted in a first color and other areas depicted in a second color;

based on the plurality of initial images and the one or more additional different images, generating at least one of a normalized difference vegetation index map, or an application map for the area;

based on the plurality of initial images and the one or more additional different images, determining an emergence map for the area;

based on the plurality of initial images and the one or more additional different images, generating at least one of a normalized difference vegetation index map, or an application map;

based on the plurality of initial images and the one or more additional different images, generating one or more agronomic recommendations for the area;

wherein the processing of the plurality of initial images and the one or more different images is performed at one or more of;

the base station, or a cloud system.

22. The system of claim 21, wherein the base station is further configured to:

identifying rows of areas above an expected greenness density on the emergence map;

partitioning areas above the expected greenness density into individual areas each corresponding to one plant of a plurality of plants and consistent with planting data;

identifying structures of at least one plant of the plurality of plants;

calculating an estimated yield from at least one plant of the plurality of plants based on the identified structures;

generating a report including the estimated yield.