Image sensor and module for agricultural crop improvement

• US 10,255,670 B1
• Filed: 02/04/2017
• Issued: 04/09/2019
• Est. Priority Date: 01/08/2017
• Status: Expired due to Fees

• Current Assignee: Dolly Y WU PLLC
• Original Assignee: Dolly Y WU PLLC
• Inventors: Wu, Dolly Y., Deutsch, Timothy A.
• Primary Examiner(s): Matt, Marnie A
• Application Number:
  US15/424,809
• Time in Patent Office: 794 Days
• CPC Class Codes: A01B 79/005 Precision agriculture
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30188 Vegetation; Agriculture
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An example machinery includes a crop management motorized vehicle having an intelligent, modularized image sensor (e.g. camera or video) system that is portable to other crop management vehicles such as a combine, planter or a tillage machine. The image sensor system includes a framework having a bank of procedures for monitoring and control of navigation, spray application, weeding, seeding, machine configuration, and so on, in real time as the machines go through a crop field throughout a crop cycle. One example implementation includes electronic circuits, with more than one set mounted on a platform that facilitates moving the setup to other agricultural machines. The framework captures, preserves and corrects the captured images for real time analysis and response, and for crop yield analysis that is correlated with the machine settings and crop management practices.

Claims

1. An image sensor system comprising:

a housing device containing a first image sensor electrically connected to a local CPU circuit and an imaging processor circuit in the housing device;

the local CPU circuit having stored crop monitoring instructions and crop-field management instructions related to different phases of a crop cycle;

the housing device mounting on an agricultural vehicle through a power pin that is electrically compatible with a voltage level of a power source on the agricultural vehicle;

the housing device including a wired and wireless communications transceiver that electrically transfers data to a signal bus of the agricultural vehicle and to a remote circuit;

the housing device having an universal adapter that mechanically and electrically attaches the housing device to a fixture on the agricultural vehicle in a harvest phase of the crop cycle and on a second type of agricultural vehicle in a second phase of the crop cycle;

the first image sensor capturing an image that is divided into a grid map containing grid members, wherein each grid member depicts a representative harvest crop area during the harvest phase;

the data including a crop field ground position that is associated with the captured image; and

the stored crop monitoring instructions including a harvest loss analysis that determines a crop yield based on color differences among the grid members in the captured image, in which an identified color cluster determines a size of a crop produce.

2. The image sensor system of claim 1, wherein the stored crop monitoring instructions further transmit a decision from the harvest loss analysis that adjusts a speed of a harvester equipment on the agricultural vehicle if the crop yield surpasses a threshold.

3. The image sensor system of claim 1, further comprising a second image sensor mounted in the housing device that captures a rearward image of crop residue, and wherein the first image sensor that captures a forward image, wherein forward is in a direction of travel of the agricultural vehicle.

4. An image sensor apparatus comprising:

a relocating modularized plug and play unit that operates on a crop management machine and a crop harvester;

the unit including multiple image sensors having fixtures that mount along a common platform;

the common platform portably mounting on a header of the crop harvester;

each of the multiple image sensors including instructions that capture images periodically at a rate proportional to a speed of processing of a crop produce at the header;

wherein a captured image is divided into a grid map, wherein each grid member depicts a representative particular header area;

wherein the captured image is coordinated with an actual ground position in a crop field; and

the unit further including a trigger system that responds based on at least one grid member having a color signal intensity surpassing a pre-selected threshold.

5. The image sensor apparatus of claim 4, wherein at least one of the multiple image sensors includes a circuit that communicates electrically with a controller of the crop harvester, and wherein the controller adjusts a cutting rate or a speed of travel responsive to a decision of the trigger system based on an estimated amount of the crop produce at the header.

6. The image sensor apparatus of claim 4, wherein at least one of the multiple image sensors is in electrical communication with a reel attached to the header, and wherein a spin speed of the reel is adjusted responsive to a decision of the trigger system based on a yield of the crop produce.

7. The image sensor apparatus of claim 4, wherein the header includes a draper header and wherein the multiple image sensors are socketed to the common platform that is attached to a back plate of the draper header.

8. The image sensor apparatus of claim 7, further comprising a program instruction that analyzes the captured image using a color difference between a cut crop and a conveyor belt of the draper header, wherein the color difference is based on differences in color among grid members in the captured image;

and wherein the program instruction estimates a crop yield using the color difference.

9. The image sensor apparatus of claim 4, wherein images from adjacent image sensors are stitched together from redundant image members, forming a redundant overlapping representation of crop rows;

and wherein a redundant image member is checked for a false trigger based on an image from a first adjacent image sensor.

10. The image sensor apparatus of claim 9, wherein the trigger system responds based on any one of the redundant members surpassing the pre-determined threshold, even if any of the adjacent image sensors has a sensor failure.

11. The image sensor apparatus of claim 4, further comprising a first program that analyzes colored grid members that align within three sigma to calculate a position of a crop row on the grid map;

and wherein the first program further determines a distance between colored grid members associated with the crop row and a grid member that is associated with a location of a planted seed.

12. The image sensor apparatus of claim 4, wherein data collected from a previous pass of a seed planter machine through the field are stored in memory circuits of the modularized plug and play unit;

and wherein the trigger system is programmed to make a decision in a subsequent pass through the field by the crop harvester.

13. The image sensor apparatus of claim 12, further comprising a second program that looks up stored data about a location of a planted seed with respect to the grid map and associates a stored location datum with an associated seed grid member;

and wherein the trigger system is further based on the at least one grid member being within a pre-determined distance from the associated seed grid member.

14. The image sensor apparatus of claim 4, wherein at least one of the multiple image sensors is positioned at an angle such that the captured image includes a representative crop area ahead of the header;

and wherein the representative particular crop area is located ahead of the header in a direction of travel of the harvester; and

wherein the representative particular crop area is based on a pre-determined height of a crop produce.

15. The image sensor apparatus of claim 4, wherein at least one of the multiple image sensors also includes a rearward looking sensor that captures a rearward image, away from the direction of travel of the harvester;

and the image sensor system further comprising a second program that analyzes the captured rearward image using a color difference among grid members in the rearward image that distinguishes between ground and a residue; and

wherein the second program determines a size of the residue and checks for harvest loss based on finding produce clusters in the residue, wherein the produce clusters include kernels or beans.

16. The image sensor apparatus of claim 4, wherein the multiple image sensors include two adjacent image sensors spaced apart on the common platform by a crop row spacing;

and the trigger system includes another program that estimates a depth distance to form a 3-D map in a forward direction using images from the two adjacent image sensors; and

wherein the trigger system analyzes the 3-D map to instruct the header to adjust its position in response to terrain changes in an upcoming crop surface.

17. An agricultural autonomous harvester having an image sensor system comprising:

a modularized plug and play unit mounted on a portable platform that transfers the unit from a crop management machine to the autonomous harvester;

the unit including multiple image sensors mounted at a regular interval along a header of the autonomous harvester;

each of the multiple image sensors programmed to record images periodically at a rate proportional to a speed of produce processing of the header;

in which each recorded image is divided into a grid map, where each grid member depicts a representative particular area of the header;

in which each recorded image is associated with a global position in a crop field; and

the unit further including a trigger system based on finding a cluster of grid members having a color signal intensity surpassing a color difference threshold compared with other grid members.

18. The autonomous agricultural harvester of claim 17, wherein at least one of the multiple image sensors also includes a rearward looking sensor that records a rearward image of a harvested ground area, away from a direction of travel of the harvester.

19. The autonomous agricultural harvester of claim 18, further comprising a second program analyzing the recorded rearward image using a second color difference and a color cluster size to detect harvest loss in the harvested ground area, wherein the second color difference determines differences in color among grid members in the rearward image.

20. The autonomous agricultural harvester of claim 17, wherein at least one of the multiple image sensors is in wireless communication with a data storage device, and wherein at least part of the captured images and the global position are uploaded to the data storage device at an end of a crop row.

21. The autonomous agricultural harvester of claim 17, wherein at least one of the multiple image sensors is in electronic communication with a central controller;

and wherein the at least one of the multiple image sensors downloads agricultural program instructions for a crop spray phase or a crop harvest phase or both phases; and

wherein the at least one of the multiple image sensors periodically uploads the recorded images and the global position.