Systems and methods for monitoring agricultural products

• US 9,939,417 B2
• Filed: 05/31/2013
• Issued: 04/10/2018
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: AGERpoint, Inc.
• Original Assignee: AGERpoint, Inc.
• Inventors: McPeek, K. Thomas
• Primary Examiner(s): Ishizuka, Yoshihisa
• Application Number:
  US13/907,147
• Publication Number:
  US 20130325346A1
• Time in Patent Office: 1,775 Days
• Field of Search: 702 2, 356 401, 47 17
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 5/0035 Measuring of dimensions of trees
• CPC Class Codes: G01N 2021/1797 in landscape, e.g. crops
• CPC Class Codes: G01N 2021/8466 Investigation of vegetal material, e.g. leaves, plants, fruits
• CPC Class Codes: G01N 21/251 Colorimeters; Construction thereof
• CPC Class Codes: G01N 33/0098 Plants or trees wood G01N33/46
• CPC Class Codes: G01N 33/025 Fruits or vegetables

Abstract

The present invention relates to systems and methods for monitoring agricultural products. In particular, the present invention relates to monitoring fruit production, plant growth, and plant vitality.

Claims

1. A plant analysis system, comprising:

a) a data acquisition component comprising a 3D laser scanner, a camera, and a survey grade global positioning satellite (“

GPS”

) receiver,

wherein said survey grade GPS receiver is configured to measure a location of said data acquisition component with a sub centimeter level of accuracy,

wherein said 3D laser scanner is configured to measure the distance between said data acquisition component and a plant utilizing waveform light detection and ranging (“

LiDAR”

), and to assemble point cloud data, by creating a plurality of three dimensional vertices, wherein each vertex represents where said plant was located relative to said 3D laser scanner, and wherein said plurality of three dimensional vertices as a whole represents the external surface of an object on said plant,

wherein said camera is configured to collect photographic data by taking an image of said plant,

b) a transport component configured to transport said data acquisition component to collect data on said plant; and

c) a processor configured to;

geo-register the point cloud data and photographic data using said location of said data acquisition component, wherein said point cloud data is geo-registered by translating said relative three dimensional coordinates of each vertex measured by said 3D laser scanner into GPS coordinates such that each three dimensional vertex of said assembled point cloud data is associated with a GPS coordinate and wherein said photographic data is geo-registered by associating each three dimensional vertex of said assembled point cloud data with a pixel of said photographic data, and wherein each three dimensional vertex of said assembled point cloud data is associated with said GPS coordinates and said pixel of said photographic data using a relational database,

determine plant color based on said photographic data,

generate classification data comprising plant stem diameter, plant height, plant volume, and plant leaf density, using said GPS coordinates of said assembled point cloud data and said plant color,

determine quality of said plant based on said plant stem diameter, said plant height, said plant volume, said plant leaf density, and together with said plant color, and

spray said plant, using a fruit tree sprayer, based on said quality of said plant, wherein said fruit tree sprayer controls when to spray, how long to spray, and what chemicals to spray on said plant based on said quality of said plant.

2. The system of claim 1, wherein said camera further comprises a thermal video camera configured to collect thermal data and said quality of said plant is further determined based on said thermal data.

3. The system of claim 2, wherein said data acquisition component further comprises a multispectral or hyperspectral sensor for gathering spectral data, and wherein said quality of said plant is further based on said spectral data.

4. The system of claim 1, wherein said processor is further configured to generate an annotated photograph of said plant.

5. The system of claim 1, wherein said plant is a fruit tree, bush, or vine selected from the group consisting of abiu, acerola, almond, amla (indian gooseberry), apple, apricot, aprium, artichokes, avocados, bael, bananas, ber (indian plum), blackberries, blood orange, blueberries, breadfruit, calamondin, cantaloupe melon, carambola (starfruit), cashew, the fruit, cherries, chestnut, chocolate, chokecherry, citron, coconuts, coffee, corn plant, crabapple, cumaquat, currant, custard-apple, dates, dewberries, dragon fruit, durian, feijoa, fig, grapefruit, grapes, guava, hazelnut, honeydew, hops, jaboticaba, jackfruit, jujube, kaffir lime, key lime, kiwifruit, kumquat, lemons, limes, loganberries, longan, loquat, lychee, mandarin, mangoes, mangosteen, medlar, morello cherry, mulberries, natal plum, nectarines, olives, oranges, papayas, passion fruit, pawpaw, peaches, pears, pecans, persimmon, pineapples, plums, pluot, pomegranate, pomelo, prune, pummel, pumpkin, raspberries, red banana, rock melon, sabine, sapodilla (chikoo), sapote, soursop, starfruit, stone fruit, strawberries, strawberry tree, sugar-apple (sharifa), surinam cherry, tamarillo, tamarind, tangelos, tangerines, tomatoes, ugli, uglifruit/uniqfruit, walnut, watermelon, a grape vine, a tomato vine, a corn plant and an apple tree.

6. The system of claim 5, wherein said fruit tree is selected from the group consisting of orange, grapefruit, mandarin, tangerine, pomelo, lemon, lime, key lime, citron, kumquat, tangelo, ugli, and blood orange.

7. The system of claim 1, wherein said processor further comprises a user interface configured to display said point cloud data, said photographic data, and said quality of said plant.

8. The system of claim 1, wherein said processor is further configured to predicted fruit yield of said plant, said predicted fruit yield determined by identifying a number of fruits on said plant based on said assembled point cloud data and said plant color.

9. A method, comprising:

a) transporting a data acquisition component, wherein said data acquisition component comprises a 3D laser scanner, a camera, and a survey grade global positioning satellite (“

GPS”

) receiver,

b) collecting waveform light detection and ranging (“

LiDAR”

) data on a plant using said 3D laser scanner while said data acquisition component is being transported by said transport component wherein said 3D laser scanner is configured to measure the distance between said data acquisition component and said plant and assemble point cloud data by creating a plurality of three dimensional vertices, wherein each vertex represents where said plant was located relative to said 3D laser scanner, and wherein said plurality of three dimensional vertices as a whole represents the external surface of an object on said plant;

c) collecting photographic data by taking an image of said plant using said camera;

d) measuring a location of said data acquisition component using said survey grade GPS receiver, wherein said survey grade GPS receiver is configured to measure location with a sub centimeter level of accuracy;

e) geo-registering said point cloud data and photographic data, wherein said point cloud data is geo-registered by translating said relative three dimensional coordinates of each vertex measured by said 3D laser scanner into GPS coordinates such that each three dimensional vertex of said assembled point cloud data is associated with a GPS coordinate and wherein said photographic data is geo-registered by associating each three dimensional vertex of said assembled point cloud data with a pixel of said photographic data,

wherein each three dimensional vertex of said assembled point cloud data is associated with said GPS coordinates and said pixel of said photographic data using a relational database;

f) determining plant color based on said photographic data;

g) determining classification data comprising plant stem diameter, plant height, plant volume, and plant leaf density using said GPS coordinates of said assembled point cloud data;

h) determining quality of said plant analyzing based on said plant stem diameter, said plant height, said plant volume, said plant leaf density, and together with said plant color, and

i) spraying said plant, using a fruit tree sprayer, based on said quality of said plant, wherein said fruit tree sprayer controls when to spray, how long to spray, and what chemicals to spray on said plant based on said quality of said plant.

10. The method of claim 9, wherein said data acquisition component further comprises a thermal imaging camera configured to collect thermal data and said quality of said plant is further determined based on said thermal data.

11. The method of claim 9, wherein said quality of said plant is based on a number of blossoms counted on said plant, wherein the number of blossoms are determined by said geo-registered vertices of said assembled point cloud data.

12. The method of claim 9, wherein said plant is a fruit tree, bush, or vine selected from the group consisting of abiu, acerola, almond, amla (indian gooseberry), apple, apricot, aprium, avocados, artichokes, bael, bananas, ber (indian plum), blackberries, blood orange, blueberries, breadfruit, calamondin, cantaloupe melon, carambola (starfruit), cashew, the fruit, cherries, chestnut, chocolate, chokecherry, citron, coconuts, coffee, corn plant, crabapple, cumaquat, currant, custard-apple, dates, dewberries, dragon fruit, durian, feijoa, fig, grapefruit, grapes, guava, hazelnut, honeydew, hops, jaboticaba, jackfruit, jujube, kaffir lime, key lime, kiwifruit, kumquat, lemons, limes, loganberries, longan, loquat, lychee, mandarin, mangoes, mangosteen, medlar, morello cherry, mulberries, natal plum, nectarines, olives, oranges, papayas, passion fruit, pawpaw, peaches, pears, pecans, persimmon, pineapples, plums, pluot, pomegranate, pomelo, prune, pummel, pumpkin, raspberries, red banana, rock melon, sabinc, sapodilla (chikoo), sapotc, soursop, starfruit, stone fruit, strawberries, strawberry tree, sugar-apple (sharifa), surinam cherry, tamarillo, tamarind, tangelos, tangerines, tomatoes, ugh, uglifruit/uniqfruit, walnut, watermelon, a grape vine, a tomato vine, and a corn plant.

13. The method of claim 12, wherein said fruit tree is selected from the group consisting of orange, grapefruit, mandarin, tangerine, pomelo, lemon, lime, key lime, citron, kumquat, tangelo, ugh, and blood orange.

14. The method of claim 9, wherein said processor further comprises a data storage, and a user interface configured to display said point cloud data.

15. The method of claim 9, further comprising determining plant health or predicted fruit ripening period, wherein said plant health or predicted fruit ripening period are determined by said geo-registered vertices of said assembled point cloud data.

16. The method of claim 9, further comprising the step of using said analyzed data to control fruit tree sprayers based on a species of said plant.

17. The method of claim 16, wherein said guiding fruit tree sprayers comprises an action selected from the group consisting of when to spray, how long to spray, and what chemicals to spray.

18. The method of claim 9, wherein said method further comprises the steps of:

gathering spectral data for said plant by a multispectral or hyperspectral sensor; and

identifying species or subspecies of said plant based on said assembled point cloud data and said spectral data,

wherein the identification of species or subspecies is performed simultaneously with the collection and geo-registration of the LiDAR data.

19. The method of claim 18, wherein said method further comprises the step of identifying a disease in said plant or a fruit on said plant, said disease manifested in a leaf, a stem, or said fruit of said plant, wherein said disease is detected by correlating said spectral data gathered for said plant with a spectral signature of a diseased plant or diseased fruit.

20. The method of claim 19 wherein said disease is citrus greening, blight, or citrus canker.

21. The system of claim 9 further comprising identifying a unique radio frequency identifier (“

RFID”

) code associated with a plant.